Memory problems

Message boards : Number crunching : Memory problems
Message board moderation

To post messages, you must log in.

AuthorMessage
Dirk Broer

Send message
Joined: 5 Nov 21
Posts: 5
Credit: 144,800
RAC: 83
Message 70 - Posted: 6 Nov 2021, 16:02:02 UTC
Last modified: 6 Nov 2021, 16:44:11 UTC

Running eight instances on a hyper threading enabled quad core results in a 'corrupted size vs. prev_size in fastbins' error.
This message comes from libc's malloc, realloc, or free, and appears to indicate either a bug in libc or memory corruption, according to here.

The machine in question has 32 GB of RAM (4GB per thread) and uses libc 2.31
ID: 70 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rysiu
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 29 Oct 21
Posts: 59
Credit: 10,300
RAC: 0
Message 77 - Posted: 7 Nov 2021, 20:32:25 UTC

The problem with memory allocation is known to us and we are working on a solution to this problem.
ID: 77 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
McShane of TSBT

Send message
Joined: 30 Nov 21
Posts: 6
Credit: 5,146,600
RAC: 0
Message 126 - Posted: 30 Nov 2021, 21:06:04 UTC

I am running an AMD Ryzen threadripper x3980 64 Cote 128thread machine with 64 Mb RAM.

Is it normal that I can only run 54 threads and the rest waiting for memory?
ID: 126 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mmonnin

Send message
Joined: 2 Nov 21
Posts: 10
Credit: 3,400,090
RAC: 2,093
Message 127 - Posted: 1 Dec 2021, 1:05:04 UTC

You have ~500mb per thread while each takes 307mb for me in Linux. Minus OS, GPU, other stuff and BOINC setting to not use all 100%, then yeah. Normal.
ID: 127 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
McShane of TSBT

Send message
Joined: 30 Nov 21
Posts: 6
Credit: 5,146,600
RAC: 0
Message 128 - Posted: 1 Dec 2021, 2:48:27 UTC - in response to Message 127.  

You have ~500mb per thread while each takes 307mb for me in Linux. Minus OS, GPU, other stuff and BOINC setting to not use all 100%, then yeah. Normal.


So, the answer seems to be to run only about a third of your available cores or just one or two project thread using app_config.xml. The trouble is that I bet most users have no idea how to set up an app_config. What a waste of resources...
ID: 128 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mmonnin

Send message
Joined: 2 Nov 21
Posts: 10
Credit: 3,400,090
RAC: 2,093
Message 130 - Posted: 2 Dec 2021, 0:16:44 UTC

Or more memory.
I try to have 1-2GB per thread for BOINC because many apps do require even more memory than 307mb.
ID: 130 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
McShane of TSBT

Send message
Joined: 30 Nov 21
Posts: 6
Credit: 5,146,600
RAC: 0
Message 132 - Posted: 3 Dec 2021, 9:26:00 UTC

My current configuration: 128 thread machine with 64Gb of RAM. I am only able to run 39 iThena threads, the rest are waiting for memory, and that is the only project that I am running.

How do I tell iThena to run only 39 threads, so that I can use the other 89 threads for other projects. ?
ID: 132 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
kotenok2000
Avatar

Send message
Joined: 7 Nov 21
Posts: 17
Credit: 56,600
RAC: 0
Message 137 - Posted: 14 Dec 2021, 21:11:16 UTC - in response to Message 132.  

<app_config>
<app>
        <name>ithena_graph</name>
         <max_concurrent>39</max_concurrent>
		   
    </app>
	
	</app_config>
ID: 137 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Memory problems

© 2019-2024 iThena. All rights reserved. | Private Policy

Page generated on 14 Jun 2024, 6:42:27 UTC in 0.4938 seconds.