Does someone has used the databricks component to push data into databricks ?
I tried to test it (writing a table with only 2 fields in string format), and it was way too slow (less than 100 records / minute). It appears that even whlle changing the batch size in the metadata and disabling the auto-commit, xDi still push data row by row to databricks.
So if anyone as a solution to push data into databricks with xDI, I will really appreciate it
Yann. courtet
Hi,
Does someone has used the databricks component to push data into databricks ?
I tried to test it (writing a table with only 2 fields in string format), and it was way too slow (less than 100 records / minute).
It appears that even whlle changing the batch size in the metadata and disabling the auto-commit, xDi still push data row by row to databricks.
So if anyone as a solution to push data into databricks with xDI, I will really appreciate it
Thanks;