Start a new topic

Load file in xDM : Load row Number to a column

Hello,


We need to load large files (approx. 10,000 lines) through the xDM user interface using an import action.

For traceability, we would like to store the source file's line number in a dedicated column (e.g., "line_number") within the golden table. Is this possible to achieve natively, or does it require a custom workaround?


Thanks for your feedback,


Jean-Christophe

1 Comment

Hello Jean-Christophe,


I need to preface my answer by saying that we do not recommend loading big data files through import in xDM.

This is even clearly documented in our official documentation:

https://www.semarchy.com/doc/semarchy-xdm/xdm/latest/Design/applications/action-sets.html#_data_size_limitations


That being said, you would like to load large files through the xDM user interface import action and keep the original source file line number in a dedicated column such as line_number in the golden table for traceability.


In xDM, this is not available as a native feature of the UI import itself.

So if you need the file line number in the golden data, you'll need a custom implementation.


I think the easier option would be to add a line_number column directly in the source file before import, then map/import it like any other business attribute.


A few points to keep in mind:

  • If you store this value in the golden table, it becomes business data from xDM’s point of view, so you should define clearly which line number should survive when multiple source records consolidate into one golden record.
  • For fuzzy-matched entities in particular, several source/master records can contribute to the same golden record, so a single line_number in GD may become ambiguous unless you define a survivorship rule or a dedicated audit design.
Login to post a comment