Jump to content


Photo

Problem with memory

node.js node-steam-tradeoffer-manager

  • Please log in to reply
6 replies to this topic

#1 PonyExpress

PonyExpress

    Member

  • Member
  • PipPip
  • 28 posts

Posted 01 May 2019 - 03:53 PM

I have noticed for a long time that my application consumes too much memory: 1500-4000mb.
Today I tried to find the reason. I just turned off different parts of the code to check.

client.on("webSession", (sessionID, cookies) => {
///////////////
//if (1==2) {
/////////////
    manager.setCookies(cookies, (ERR) => {
        if (ERR) {
            console.log($error, currentTime() + " [  ACCOUNT  ] [       LOGIN       ] An error occurred while setting cookies: " + ERR);
        } else {
            console.log(currentTime() + " [  ACCOUNT  ] [       LOGIN       ] Websession created and cookies set");
        }
    });
///////
//}
///////
    community.setCookies(cookies);
    community.startConfirmationChecker($SECOND * 10, CONFIG.IDENTITYSECRET);
    // ....

});

When I turned off manager.setCookies i saw it:

 

start WITHOUT -- manager.setCookies:

23:22 [   DEBUG   ] [     INVENTORY     ] Loading inventory for checking cardsets...
23:23 : 197.4921875(process.memoryUsage().rss / 1048576)
23:23 [   DEBUG   ] [     INVENTORY     ] Cardsets loaded: 3438/1433
23:23 : 253.53125(process.memoryUsage().rss / 1048576)
23:23 : 253.6640625(process.memoryUsage().rss / 1048576)
23:23 : 254.43359375(process.memoryUsage().rss / 1048576)
23:23 : 255.1875(process.memoryUsage().rss / 1048576)
23:23 : 255.84765625(process.memoryUsage().rss / 1048576)
23:24 : 256.328125(process.memoryUsage().rss / 1048576)
23:24 : 257.1953125(process.memoryUsage().rss / 1048576)
23:24 : 257.22265625(process.memoryUsage().rss / 1048576)
23:24 : 257.9453125(process.memoryUsage().rss / 1048576)
23:24 [   DEBUG   ] [     INVENTORY     ] Loading inventory for checking keys amount...
23:24 : 213.578125(process.memoryUsage().rss / 1048576)
23:24 : 216.2734375(process.memoryUsage().rss / 1048576)
23:25 : 220.11328125(process.memoryUsage().rss / 1048576)
23:25 : 220.46484375(process.memoryUsage().rss / 1048576)
23:25 : 228.640625(process.memoryUsage().rss / 1048576)
23:25 : 228.11328125(process.memoryUsage().rss / 1048576)
23:25 : 229.203125(process.memoryUsage().rss / 1048576)
23:25 : 229.60546875(process.memoryUsage().rss / 1048576)
23:26 : 229.84765625(process.memoryUsage().rss / 1048576)
23:26 [  MESSAGE  ] [ ................. ] abcd
23:26 : 229.09765625(process.memoryUsage().rss / 1048576)
23:26 : 181.03125(process.memoryUsage().rss / 1048576)
23:26 : 181.59765625(process.memoryUsage().rss / 1048576)

start WITH -- manager.setCookies:

23:28 [   DEBUG   ] [     INVENTORY     ] Loading inventory for checking cardsets...
23:28 : 185.8125(process.memoryUsage().rss / 1048576)
23:28 [   DEBUG   ] [     INVENTORY     ] Cardsets loaded: 3438/1433
23:28 : 290.54296875(process.memoryUsage().rss / 1048576)
23:28 : 1303.80078125(process.memoryUsage().rss / 1048576)
23:28 : 1290.59375(process.memoryUsage().rss / 1048576)
23:29 : 1283.73828125(process.memoryUsage().rss / 1048576)
23:29 : 1262.2578125(process.memoryUsage().rss / 1048576)
23:29 : 897.84375(process.memoryUsage().rss / 1048576)
23:29 : 873.96484375(process.memoryUsage().rss / 1048576)
23:30 : 1661.671875(process.memoryUsage().rss / 1048576)
23:30 [   DEBUG   ] [     INVENTORY     ] Loading inventory for checking keys amount...
23:30 : 1653.04296875(process.memoryUsage().rss / 1048576)
23:30 : 1659.85546875(process.memoryUsage().rss / 1048576)
23:30 : 1663.953125(process.memoryUsage().rss / 1048576)
23:30 : 1674.36328125(process.memoryUsage().rss / 1048576)
23:30 : 1705.48046875(process.memoryUsage().rss / 1048576)
23:31 : 1715.796875(process.memoryUsage().rss / 1048576)
23:31 [  MESSAGE  ] [       .  .  .    ] asdf
23:31 : 1695.4921875(process.memoryUsage().rss / 1048576)
23:31 : 1709.73046875(process.memoryUsage().rss / 1048576)

I also noticed that my folder 'pool_data' has a size about 40mb / 40 000 files (I don't know if this is normal or not)

 

I would like to know where I can continue to look for the error to reduce memory usage.



#2 Dr. McKay

Dr. McKay

    Developer

  • Administrator
  • 2,140 posts

Posted 01 May 2019 - 11:14 PM

fi0rI.png



#3 PonyExpress

PonyExpress

    Member

  • Member
  • PipPip
  • 28 posts

Posted 02 May 2019 - 06:08 AM

Dr. McKay, I have already tried to delete community.startConfirmationChecker but it gave no results in using memory.



#4 PonyExpress

PonyExpress

    Member

  • Member
  • PipPip
  • 28 posts

Posted 02 May 2019 - 06:38 AM

To make it more visual, I deleted almost everything and checked:

var SteamTotp = require("steam-totp");
var TradeOfferManager = require("steam-tradeoffer-manager");
var SteamCommunity = require("steamcommunity");
var SteamUser = require("steam-user");

var CONFIG = require("config.js");

var client = new SteamUser();
var community = new SteamCommunity();
var manager = new TradeOfferManager({
          "steam": client,
          "community": community,
          "language": "en",
          "pollInterval": "10000",
          "dataDirectory": CONFIG.POOLDATA.DIR, // "./pool_data"
          "savePollData": CONFIG.POOLDATA.SAVE // true
      });

client.logOn({
    accountName: CONFIG.USERNAME,
    password: CONFIG.PASSWORD,
    twoFactorCode: SteamTotp.getAuthCode(CONFIG.SHAREDSECRET),
    identity_secret: CONFIG.IDENTITYSECRET,
    shared_secret: CONFIG.SHAREDSECRET,
    logonID: 1
});

client.on("loggedOn", () => {
    console.log("currentTime()" + " [  ACCOUNT  ] [       LOGIN       ] Name: " + CONFIG.USERNAME + " (" + client.steamID + ")");
    client.setPersona(1);
});

client.on("webSession", (sessionID, cookies) => {
    manager.setCookies(cookies, (ERR) => {
        if (ERR) {
            console.log("currentTime()" + " [  ACCOUNT  ] [       LOGIN       ] An error occurred while setting cookies: " + ERR);
        } else {
            console.log("currentTime()" + " [  ACCOUNT  ] [       LOGIN       ] Websession created and cookies set");
        }
     });
    community.setCookies(cookies);
});

community.on("sessionExpired", () => {
    console.log("currentTime()" + " [  ACCOUNT  ] [       LOGIN       ] Session expired. Relogging...");
    client.webLogOn();
});
// 800-1400 Mb memory, ~30-50% cpu
// w/o manager.setCookies: 72-107 Mb memory, 0.2-1.5% cpu

Edited by PonyExpress, 02 May 2019 - 06:44 AM.


#5 Dr. McKay

Dr. McKay

    Developer

  • Administrator
  • 2,140 posts

Posted 02 May 2019 - 10:15 PM

Memory usage is pretty normal on accounts with a lot of trade offers. The module caches asset descriptions in order to avoid slamming the API and using up your quota of requests on asset description retrieval.

 

If you want to minimize memory usage, you need to omit a language in the constructor and keep track of whatever data you might need yourself.


  • PonyExpress likes this

#6 PonyExpress

PonyExpress

    Member

  • Member
  • PipPip
  • 28 posts

Posted 03 May 2019 - 04:37 AM

Dr. McKay, Thank you, at first glance it helped reduce memory consumption to 250-900 Mb.

 

 

 

If I open bot trade offers page, I see that the offers are not cleared until i see them.
For example, if I open the page 'Sent offers', first time I will see:
1. [Exchange rejected May 3. 2019]
2. [Exchange accepted May 3. 2019]
3. [Exchange accepted May 3. 2019, Delay: 12 days]
After uptade this page I will see:
1. [Exchange accepted May 3. 2019, Delay: 12 days]
 
I need to worry about it and somehow clean them or not?


#7 Dr. McKay

Dr. McKay

    Developer

  • Administrator
  • 2,140 posts

Posted 03 May 2019 - 05:13 PM

Offers will disappear on their own after a while.







Also tagged with one or more of these keywords: node.js, node-steam-tradeoffer-manager

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users