GTA Online remains a popular (and incredibly profitable) game seven years after launch, thanks to the steady influx of new content, but one thing Rockstar seems unable to improve is the game’s famously long load times. Over the weekend, an enterprising developer called t0st finally discovered why GTA Online takes so long to load – even on machines with fast processors and storage such as the PlayStation 5 and PC – and fixed those issues, reducing his load times by 70 per cent.
The blog written by t0st explaining the issues and fixes is brilliant, complete with excellent MSPaint illustrations, but it’s a little hard to follow if you don’t have programming experience. I’ll try to summarise it as best I can!
So: after struggling through a six minute load for GTA Online on his mid-range PC, t0st opened Task Manager the next time he loaded up the game and noticed something odd: after the one minute mark, his computer’s CPU usage spiked up dramatically, but storage and network usage were nearly non-existent. That suggested that the lengthy load times weren’t caused by Rockstar’s servers or data being read off a drive – instead, something was running on the CPU. Something that required a ton of processing to complete, and only used a single thread.
Armed with this knowledge, t0st used a series of programming and debugging tools to discover two major issues.
First, the game was reading in a text file of all purchasable items in the game – and after every one of 63,000 items, it counts every character in the 10MB text file afresh. Doing this count once is no big deal, but doing it 63,000 times adds up to a whole lot of wasted CPU time.
Second, to prepare all of the item data that’s been read in, the game records both the data associated with that item (eg its name, price, category, stats) and a hash of that item (essentially a calculated ‘fingerprint’ that uniquely identifies it). Each time the game stores an item from the list – which, remember, happens 63,000 times – it checks the hash value of the item being stored against the hash value of every other item that’s already been stored.
At first, this doesn’t take much time, but as the number of items successfully loaded into the game increases, this check takes longer and longer and longer. In all, t0st estimates that 1,984,531,500 checks take place (nearly two billion!), again taking up a ton of CPU time. The game does this to make sure that there are no duplicate items in the final list, but given that the list is completely empty to start with and the file being loaded in has no duplicates, the check is essentially pointless.
To solve the issues, t0st wrote his own code that overwrites some of the game’s functions. To solve the ‘reading in items’ issue, he created a basic cache, which calculates the length of the list of items once, then returns that same value without actually doing the calculation again whenever the length is asked for by Rockstar’s code. That cuts the number of times that the check needs to be done from 63,000 to one, saving a massive amount of unnecessary work.
The second fix is even easier. As t0st reckoned that there’s no need to check for duplicate items, his code just inserts the new item directly, without performing the hash check at all. That means none of the nearly two billion checks need to take place, and the CPU can rocket through the process.
With both fixes in place, GTA Online loads way faster. On t0st’s PC, avoiding the duplicate items check cuts the load time from six minutes to four-and-a-half minutes, and adding in the item loading fix as well brings this down further, to just one minute and 50 seconds. That’s a 69.4 per cent reduction in load times, which is absolutely incredible given that it required modification of only two functions.
So what happens now?
Update (5PM): We caught up with t0st to ask him why he thought these seemingly very obvious issues might have persisted for so long. He pointed out that he doesn’t have game dev experience and the debugging tools only give him a small window to view what’s happening in the code, so he doesn’t know for sure. However, he pointed to several comments from users of the Hacker News board as theories that he liked. For example, the code may have been only been tested on smaller item lists, so the extent of the problem may not have been initially obvious. It’s also possible that the team maintaining GTA Online might have few of the original programmers, so the inner workings of the code might not be well understood. He also touched on the company politics angle; generally management needs to justify any programming time spent, so things like optimising load times might not have received the necessary investment to carry out. It’s all interesting food for thought, and I hope we’ll one day have a better insight as to why and how this situation came to pass.
Well, if you’re an experienced games programmer with the right tools, you can download the source code here and try the fix for yourself. Note that modifying game functions while the game is running in this way is classic hacking behaviour, so proceed with caution lest you end up with a permanent ban.
For everyone else, your best bet is to hope that Rockstar implements the fixes in a future GTA patch. The relative simplicity of the fix and the magnitude of the time savings ought to mean it’s at least worth investigating, especially for players on console or PCs with older AMD processors. As t0st himself put it:
“If this somehow reaches Rockstar: the problems shouldn’t take more than a day for a single dev to solve. Please do something about it :<
“You could either switch to a hashmap for the de-duplication or completely skip it on startup as a faster fix. For the JSON parser – just swap out the library for a more performant one. I don’t think there’s any easier way out.
ty <3″
Given the attention that this story has received since it was published on 28th February, you’d hope that Rockstar would at least offer a response, given how much of a bugbear load times have been for GTA Online over its long history. We’ll update this story if Rockstar do speak publicly on the issue.
