Fetching by batch (100k+ records)
I have a angular app with django backend . On my front-end I want to display only seven column out of a identifier table. Then based on an id, I want to fetch approximately 100k rows and 182 columns. When I am trying to get 100k records with 182 columns, it is getting slow. How do I speed up the process? Now for full context, i am currently testing on localhost with 16gb ram and 16 cores. Still slow. my server will have 12gb of rams and 8 cores.
When it will go live., then 100-200 user will login and they will expect to fetch data based on user in millisecond.