krogon500 Posted May 22 Report Posted May 22 I want to get one field from product info for almost all Steam apps (around 200k), and save it to my DB. My code looks something like that: const client = new SteamUser(); /*...*/ const appIdList = []/*Just an array with app ids*/; const appListForDB = []; const chunkSize = 1000; for (let i = 0; i < appIdList.length; i += chunkSize) { const chunk = appIdList.slice(i, i + chunkSize); let result = await client.getProductInfo(chunk, [], false); const apps = result.apps; for (const key in apps) { const appInfo = apps[key].appinfo; /*Populating list for DB, it has an app id and a game executable string*/ } } /*Logging Off*/ client.logOff(); /*Inserting values to DB*/ DBUtils.insertValues(appListForDB); I split app id list into chunks, but the RAM usage is just building up after each chunk. So no garbage collected. I even tried to remove all DB related stuff, and RAM usage is still building up (so I think it's the "SteamUser" fault). Is it normal behavior and I just need more RAM, or something can be done to make it work within ~200 MB RAM usage? Quote
Dr. McKay Posted May 23 Report Posted May 23 Garbage is only collected periodically. If you want to fetch that much data in Node, you're going to need more than 200 MB of RAM. Quote
krogon500 Posted May 24 Author Report Posted May 24 Okay. But I did the same in C# with SteamKit2 and RAM usage is around 400 MB (against 2 GB+ with your lib in Node). I guess, Node is different with RAM usage? I don't know much about Node and JS in general, so my question may sound dumb... Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.