• gm_heap
    23 replies, posted
If you don't know what a pointer is, or what the heap is, don't bother with this thread. So yea, this module lets you allocate and deallocate memory from the heap. Download: [url]http://www.fortfn.co.uk/files/gm_heap.dll[/url] Source: [url]http://pastebin.com/Nm4tdyJn[/url] Usage: [lua]local ptr = heap.Allocate( 10 ) -- Allocate 10 bytes heap.Set( ptr, 0, 10 ) -- Set all of our newly allocated memory to 0 for i=1, 10 do heap.SetByte( ptr+i - 1, i ) -- Set each byte to a value end for i=1, 10 do print( heap.GetByte( ptr+i - 1 ) ) -- Print it end local ptr2 = heap.Allocate( 10 ) -- Allocate another 10 bytes heap.Set( ptr2, 0, 10 ) -- Set all of our newly allocated memory to 0 heap.Copy( ptr, ptr2, 5 ) -- Copy 5 bytes across print( "---" ) heap.Deallocate( ptr ) -- Deallocate our original 10 bytes for i=1, 10 do print( heap.GetByte( ptr2+i - 1 ) ) -- Print it end heap.Deallocate( ptr2 ) -- Deallocate our last 10 bytes[/lua] [code]1 2 3 4 5 6 7 8 10 --- 1 2 3 4 5 0 0 0 0 0[/code] Now of course, using a module for storing an array of 10 bytes is completely stupid, but suppose you want to store an array of 1,000,000 bytes (~1 mb of data). Lua will not offer you the speed to manage, access and set the data in a table of such size but this module should do.
Can we get perf times for both a 1mb table(4 bytes per element) and 1mb of data with your module. And perhaps 10mb, 100mb and 500mb tests. [editline]02:32AM[/editline] Also you need to encapsulate the pointers, so the do bounds checking or something.
This could be very unsafe if it wanted to be.
Awsome module, but i cant find any use for it [editline]11:18AM[/editline] Actaully lets store the players data in the heap! :pseudo:
As haza said we need proper testing before :)
It's only as unsafe as the scripts which use it. And I'll do some speed tests later.
I will use this for storing items in my gamemode, as i will have around.... 5k+ items placed on the map it will be a good test of this [editline]04:07PM[/editline] Deallocating seems to take about... forever on 1mb data
I did some basic testing. [lua] require( "heap" ) local B = 1 local KB = B * 1024 local MB = KB * 1024 local cnt = 10 * MB local function cmdTestLuaTable( ply, com, args ) debug.sethook() local start = os.time() local tbl = {} for i=1, cnt do tbl[i] = i % 255 end local o = 0 for i=1, cnt do o = o + tbl[i] end print( o ) print( "Lua table took " .. (os.time()-start) .. " seconds to test!" ) end concommand.Add( "heap_testluatable", cmdTestLuaTable ) local function cmdTestHeapTable( ply, com, args ) debug.sethook() local start = os.time() local ptr = heap.Allocate( cnt ) local Set = heap.SetByte local Get = heap.GetByte for i=1, cnt do Set( ptr + i - 1, i % 255 ) end local o = 0 for i=1, cnt do o = o + Get( ptr + i - 1 ) end print( o ) heap.Deallocate( ptr ) print( "Heap array took " .. (os.time()-start) .. " seconds to test!" ) end concommand.Add( "heap_testarray", cmdTestHeapTable )[/lua] [code] ] heap_testluatable not enough memory ] heap_testarray 1331684080 Heap array took 5 seconds to test! [/code] Seems like the limit for the amount of data to be stored in a lua table is between 1mb and 10mb. Both loops cause Garry's infinite loop detector to be triggered off (hence debug.sethook). I guess there's no way to make looping ~1,000,000,000 times in Lua any faster (without LuaJIT). So I think this module is only good if you need to store ALOT of memory and need to access it fast (like mine and overv's minecraft in gmod projects). I'm not gonna bother going higher than 10mb since obviously you can't test lua tables, and I'd get bored waiting.
I just got a hl2.exe error after testing that one, its pretty unstable, i think its the deallocating
I hope you don't mind, but I was already working on a module that will probably depreciate this, so don't be offended if it does :saddowns:.
[QUOTE=Gbps;23733860]I hope you don't mind, but I was already working on a module that will probably depreciate this, so don't be offended if it does :saddowns:.[/QUOTE] Go right ahead, my implementation is pretty crappy to be fair. [editline]08:08PM[/editline] [QUOTE=Tobba;23729007]I just got a hl2.exe error after testing that one, its pretty unstable, i think its the deallocating[/QUOTE] Can you show me your code?
All i did was some heap.Allocate some heap.SetByte and some heap.Deallocate, it crashes when you Dealloc it, your example crashes too, i think its your de-allocation that crash it
It could be, but my example works fine on my PC. Are you testing on a win32 machine?
You should use malloc and free. new and delete are probally performing rtti checks.
This module seems rather useless, because unless you're going to read a value from memory, Lua can do the exact thing. It abstracts the ability to call malloc/free by giving you garbage collected values like tables. If you want to store 5000 items use a table with unique ids as keys. It will be many times faster than any system you could possibly create with the scope of this module. If you're worried about memory constraints, move it into SQL and memcached so you can cache item data and anything else. Also, as for your benchmark about not being able to allocate the table, it's because of the way tier0 handles memory allocation. It's not optimized for the way Lua works, which was one of the issues being investigated. (Many allocations of small objects will take up larger buckets, large allocations will fail because of this. The bucket system isn't exactly suited for Lua)
This is better in some cases when lua runs out of memory, its just that its.... crashy
Yeap, the source engine mallocs hunks(heap chunk). It then deals out small proportions of that memory. This is generally faster because we don't have to do a syscall, wait for the operating system to allocate a virtual block of memory and return(and all the extra descriptors placed on that memory etc etc).
[QUOTE=AzuiSleet;23741671]This module seems rather useless, because unless you're going to read a value from memory, Lua can do the exact thing. It abstracts the ability to call malloc/free by giving you garbage collected values like tables. If you want to store 5000 items use a table with unique ids as keys. It will be many times faster than any system you could possibly create with the scope of this module. If you're worried about memory constraints, move it into SQL and memcached so you can cache item data and anything else. Also, as for your benchmark about not being able to allocate the table, it's because of the way tier0 handles memory allocation. It's not optimized for the way Lua works, which was one of the issues being investigated. (Many allocations of small objects will take up larger buckets, large allocations will fail because of this. The bucket system isn't exactly suited for Lua)[/QUOTE] It's not useless when you want to store 250,000+ items.
Cool, this is just like ( warning ) [SP] the cheat engine [/SP] and you can ( Warning ) [SP] speed hack by changing the speed values and update timers [/SP] but sadly i think garry or valve fixed it :(:
[QUOTE=Ningaglio;23778596]Cool, this is just like ( warning ) [SP] the cheat engine [/SP] and you can ( Warning ) [SP] speed hack by changing the speed values and update timers [/SP] but sadly i think garry or valve fixed it :(:[/QUOTE] What? If you want to use this to cheat, you still have to find the memory addresses of stuff you want to change and get them into lua. Using cheat engine would be easier than using this to cheat.
Even with free and malloc it crashes, im running 32 bit vista (probably vista fucks it up)
FINALLY i figured it out, you have to add a NULL at the end of the memory you allocated before you deallocate it or it will fuck up and unallocate a few thusand bytes before it comes to a NULL char [editline]12:38PM[/editline] Wait a little. doing more tests [editline]12:39PM[/editline] Confirmed
[QUOTE=Tobba;24774905]FINALLY i figured it out, you have to add a NULL at the end of the memory you allocated before you deallocate it or it will fuck up and unallocate a few thusand bytes before it comes to a NULL char [editline]12:38PM[/editline] Wait a little. doing more tests [editline]12:39PM[/editline] Confirmed[/QUOTE] That's weird. Oh well. I started making a new "safe" version of this ages ago, but never got it finished.
you can just specify the size of what it shall deallocate, like delete [Lua()->GetInterger(2)] ptrMem;
Sorry, you need to Log In to post a reply to this thread.