OHDSI Home | Forums | Wiki | Github

White Rabbit Scan Failure for Large Number of Records

Hello, I tried to scan a csv file of lab data with 1.7 million records using White Rabbit. The scan report failed to read the file properly and present the normal scan results. After I reduced the dataset to a smaller set of < 100K records, the scan was successful. I searched if there is a limit on the number of records in the data source for White Rabbit, and I didn’t find any relevant information, so I’m posting my question here. So is there an upper limit in terms of the number of records in a csv file that White Rabbit will be able to scan successfully? Thank you!

Edit: I figured out what went wrong with the scan, and WR works fine now.

-Tao

t