Jump to content


Photo
Discussion

Possible ram usage improvement in module

node.js node-steam-tradeoffer-manager

Best Answer Dr. McKay , 21 June 2019 - 12:57 AM

This is already done in the module via persisting to local disk with the dataDirectory option (which you can redirect to go anywhere you want, including MongoDB). Update to v2.10.0 and you can override the maximum size of the asset cache, but beware that doing so will negatively impact performance as more and more items will need to be retrieved from the disk for every poll.

 

In practice, for the best performance, you will need more RAM the more trades you make.

Go to the full post


  • Please log in to reply
9 replies to this topic

#1 Blue

Blue

    Member

  • Member
  • PipPip
  • 26 posts
  • LocationBrazil

Posted 21 June 2019 - 12:15 AM

I was doing some tests in one of my projects that use this module, and i noticed that de polling sometimes requests to much ram, after get further the module in github, the "problem", seems to be that to much information is cached in the LeastUsedCache(), this happens only when there is a that bot account has to many items i suppose..

 

so having that in mind, i have a question that can possibly lower the ram usage, what if we use a mongoDB or something alike to store that data intead(as an option to implement of course), this would only be useful in some cases like mine, but it might improve a little the ram usage, and also would be possible to store more data for more time if needed..

 

i could have did it myself and test it to see if is worth it, but i'd rather see you guys thoughts first, and also the possibility of this to be implemented in the module by the @Doctor himself, if the results seems to be good



#2 Dr. McKay

Dr. McKay

    Developer

  • Administrator
  • 2,137 posts

Posted 21 June 2019 - 12:57 AM   Best Answer

This is already done in the module via persisting to local disk with the dataDirectory option (which you can redirect to go anywhere you want, including MongoDB). Update to v2.10.0 and you can override the maximum size of the asset cache, but beware that doing so will negatively impact performance as more and more items will need to be retrieved from the disk for every poll.

 

In practice, for the best performance, you will need more RAM the more trades you make.



#3 Blue

Blue

    Member

  • Member
  • PipPip
  • 26 posts
  • LocationBrazil

Posted 21 June 2019 - 01:06 AM

so basicaly if i down a bit the asset cache values, i would use less ram, but the polling would be done slower? this doesnt sound that bad, but how badly this will impact the performace?



#4 Dr. McKay

Dr. McKay

    Developer

  • Administrator
  • 2,137 posts

Posted 21 June 2019 - 01:20 AM

If you're going to lower the max items value, I would also configure a dataDirectory so that excess items can be stored locally on the disk instead of having to consult Steam every time. Performance shouldn't be impacted too heavily, but it may be noticeable. You'll definitely minimize your impact by configuring a dataDirectory.



#5 Blue

Blue

    Member

  • Member
  • PipPip
  • 26 posts
  • LocationBrazil

Posted 21 June 2019 - 01:51 AM

Im using savePollData:true on the constructor, so my dataDirectory  is already seted right?



#6 Dr. McKay

Dr. McKay

    Developer

  • Administrator
  • 2,137 posts

Posted 21 June 2019 - 01:55 AM

As long as you didn't set dataDirectory: null you should be fine.



#7 Blue

Blue

    Member

  • Member
  • PipPip
  • 26 posts
  • LocationBrazil

Posted 21 June 2019 - 02:45 AM

ok cool, what about the steamcommunity module, this proccess is the same about the getInventoryContents? i saw that you made a temp var to load the target inventory, but if the target inventory is to big it can pretty much overload the ram as well right? would be cool to have something alike to keep control of ram usage if it doesnt exist yet, also thanks for the quick responses =D



#8 Dr. McKay

Dr. McKay

    Developer

  • Administrator
  • 2,137 posts

Posted 21 June 2019 - 01:52 PM

I have no idea what you're talking about.



#9 Blue

Blue

    Member

  • Member
  • PipPip
  • 26 posts
  • LocationBrazil

Posted 21 June 2019 - 06:10 PM

// A bit of optimization; objects are hash tables so it's more efficient to look up by key than to iterate an array
	var quickDescriptionLookup = {};

	function getDescription(descriptions, classID, instanceID) {
		var key = classID + '_' + (instanceID || '0'); // instanceID can be undefined, in which case it's 0.

		if (quickDescriptionLookup[key]) {
			return quickDescriptionLookup[key];
		}

		for (var i = 0; i < descriptions.length; i++) {
			quickDescriptionLookup[descriptions[i].classid + '_' + (descriptions[i].instanceid || '0')] = descriptions[i];
		}

		return quickDescriptionLookup[key];
	}

This part of your node-steamcommunity, i meant the "quickDescriptionLookup", if the target inventory has to many descriptions, it could overload ram right?

 

like for instance some target inventory with 100k items or something..


Edited by Blue, 21 June 2019 - 06:10 PM.


#10 Dr. McKay

Dr. McKay

    Developer

  • Administrator
  • 2,137 posts

Posted 22 June 2019 - 08:15 PM

Sure, I guess, but there's no way around that. You just need to have enough memory.







Also tagged with one or more of these keywords: Discussion, node.js, node-steam-tradeoffer-manager

2 user(s) are reading this topic

0 members, 2 guests, 0 anonymous users