Survive the Wild early April update

Hello again everyone, it’s time for another dollop of Survive the Wild development news! We’ve had a productive couple of weeks touching on a lot of random systems, so without further ado lets just get right into it!

Various lag reduction experiments

Quite a bit of time these last 2 weeks has been spent trying to fix some weird lag that was taking place in the new object system involving scanning for nearby objects and filtering them by certain criteria. For quite a while the system worked pretty well, but at some unknown point something began falling apart, and it was discovered that firing many weapon projectiles very fast would cause an unpleasant amount of lag. On the surface this may seem fine as players could never fire weapons nearly as fast as we were doing so for testing on the beta server, but the projectile impact scanning code used the same core systems as everything from tile fetching to the broadcasting of nearby object sounds. The faster we can make those systems, the better all around for everyone.

As such, intermittently as I’ve been doing other things described below, I’ve been trying to tackle some of the unexpected lag issues. I knew that the way I coded my system, it should absolutely not lag like that for the small number of nearby objects that were causing the lag. My adventures in this matter are long and complex and would probably bore most people reading this. Suffice it to say that early today I finally made a breakthrough, and projectiles became just about as lag free as we want them to be as well as the servers responsiveness subtly increasing, just like I wanted. We will be fighting with lag of some sort until the end of Survive the Wild, everything must be perfect for server stability. Though it took me a while to track it down, I realized that after I was querying all objects generally within the scan area which was already working just as it should, I was doing too much additional scanning/checking/filtering in the wrong order before returning objects. It’s a bit tricky to explain here and I don’t know if many would want me to, but regardless I figured out how I should be doing this filtering properly earlier today and the results of it are already showing!

It’s somewhat less satisfactory news regarding the pathfinder specifically. I talked about writing a distance score or distance desperation function to help npcs figure out what food item to pathfind to. I did try writing this function shortly after finishing up the last blog post, but that function’s performance itself is making it not worth it, do to the need to query the desperation factor of every tile in the straight line path to every nearby item. I’m still thinking about whether there is a way I can fix my attempt at that function, or whether a new aproach is needed. I’m going to let the server run some over the weekend with the new general lag fixes and see what things look like. Even though this idea didn’t yield any fruit, I think the general lag improvements may have actually compensated to a significant degree for it, because of the fact that these improvements involve the system that queries for tiles while pathfinding, a system that is called into sometimes hundreds or thousands of times a second.

Ending on a high note where lag improvements are concerned though, I did do a couple of other things. First, I heavily improved the profiler system in NVGT. I ended up being required to do this in order to track down the issue I mentioned above. The result though is that I can now get much more accurate information about how long certain functions are taking to execute, and thus what functions I need to refactor or improve when solving laggy situations in the future. The other thing I did was to improve the interleaving of our timer wheel. This basically means deciding how much processing power should be spent executing object timers and how much should be spent handling network packets from players and other general server upkeep. Previously animals were getting a little slow after a while which I really don’t want, however that has now been drastically improved upon as unless a timer starts risking the stability of the server, I’m now letting many more timers execute each iteration of the server’s main loop. This again could start getting complicated so I’ll stop here, but the point is that you should get ready because animals are going to be coming after you quickly indeed when the game comes out if we have any say in the matter! It will be more clear how successful these improvements were after some extended tests, but I’m highly confident that things are running quite a bit better than they were before. Finally the last notable thing I can think of regarding lag improvements is that the client can now update the sound positions of objects much more quickly than before when there are many sounds currently playing. Previously if an object moved, a loop was made through every sound currently playing to check if it should update it’s position given an owner. Now we only need to loop through those sounds pertaining to the object that moves, significantly reducing the amount of checks and comparisons required to simply update a sound. Hopefully this will really be seen on lower end computers. A lot of time over these last 2 weeks has been spent insuring that the new update is as stable as possible, and I’d say we’ve been making great progress!

Administration user interface updates

In the last post I mentioned that I had 4 more controls to add to that giant character modification panel. While I still need to tackle quests and this minor variables dictionary, I’m happy to say that the slash commands for jailing and unjailing players as well as for warning them have now been removed and converted to controls in that panel! It’s more than that though. In regards to warnings, that system has been rewritten. Previously, warnings were just a sound and a message that appeared in the players buffer. Furthermore warnings were character based, not account based. Now, a moderator has the ability to warn a users entire account, and when they do, that user will see a dialog on any of their characters when they log in. The user must dismiss this dialog after manually accepting the warning before they can continue playing the game. Previously a player could try telling an administrator that they did not see the warning or somehow missed it, this is no longer the case. It is logged in the users account history both when the warning appears and when the user confirms that they understand it.

As for jailing, that’s also received some work. First, the jail database was converted to SQLite like all of the other data. I wasn’t planning on doing this originally until I realized how easy it would be, and got it done in 45 minutes or so. I also spent some more time generally modernizing the jail code while I was at it. Previously all of the cell locations were actually hardcoded in the server application, meaning that admins couldn’t completely modify the jail. Now the server actually reads the jail map to determine where all possible cells are, as should have been done ages ago. Then I started on the user interface, and got it basically working! Though it needs some tweaking and some convenient features, it is now possible to jail players from the giant administration panels as well as unjail them. Administrators have finally had granted one of their greatest wishes, the ability to bulk jail an account all at once! Previously an administrator would spend literally minutes enumerating through all characters on someone’s account and manually typing the commands and jail reasons for every single character one after another to get a players account jailed. This is finally no longer the case and nobody can even imagine how much time it’s going to save! At this point there isn’t much left regarding modifications to the jail system, and I can’t imagine it’ll take more than another hour or 2 of work.

It’s also worth noting that the rank permission switches that control access to the character panels are now set up, so moderators actually finally have something to look at!

There were some other improvements to admin UI as well, such as a couple of bugs fixed in some dialogs, and the spending of a couple hours generalizing the code for the dialogs that allow admins to modify in-game scripts so that they will also be able to be used to modify other game datafiles without needing slash commands to do it, which was tested by also getting rid of the slash commands for the setting of map data. Though at this point I need to be careful how much I focus on less vital parts of the administration UI in favor of getting this update out faster, the more UI we can get done the less confusion, mistakes, questions and redundant documentation writing will need to take place as a result. Considering that I’ve now gotten the bulk of the aforementioned necessities complete though, I think I’m satisfied with my progress regarding the administration user interface over these last 2 weeks!

A few long time issues resolved

We fixed just a couple of things that have needed attention for quite a while now. One major thing, and 2 minor ones related to chatting.

The major long time bug we fixed had to do with free characters spending credits and points. This one was quite major actually and I’m surprise I hadn’t known about it before now. There is a chance that this bug was even a thing when you had to type slash commands to spend points and credits, though I’m not sure about that as old code indicates that the previous behavior back then may have been to just allow the purchases even beyond the free item limit. However at some unknown point in the past, it seems that a very bad situation was introduced where a free character could try purchasing say 200 of any item, but though they would lose 200 items worth of credits or points, they would only receive up to 100 of the item they purchased because of the free character limit! This is actually quite severe and I have to issue a sincere apology to anybody who has ever lost any stw credits or points because of this. Somehow I genuinely did not know about this issue until a beta member found and reported it days ago, where it was then pretty quickly fixed. This just makes it clear why a better bug reporting system has always been needed for Survive the Wild, luckily we finally have one now and it’s benefits are already clearly showing! If in the future you are playing the game and you feel like something weird like this happens to your character, please make sure you open not only a bug report, but also a support ticket so that staff can make sure you don’t end up short of any points or credits! Again I deeply apologize to anybody who was effected by this and who did not ask for help restoring their credits, I do believe that with the new bug reporting systems there will never be a critical bug like this again that exists for significant periods of time. It wasn’t that difficult to fix, but unfortunately I did not know about it until recently!

With that unpleasantness out of the way, lets briefly talk about 2 other rather relieving improvements having to do with chatting. First, if you have a partially typed message in a conversation and then you switch to a different one, the partially typed message is stored in a buffer in the conversation that was just deselected. Then when that conversation is activated again, the message text field is populated with the partially typed message from before, insuring that there is no longer any chance to send a partially typed message to the wrong person! Speaking of that, the second improvement is that the message text field now contains the name of the conversation you are messaging in, so that now you have simply to read the label of the message text field by pressing alt+m to verify that you’re talking to the intended recipient. While all of these fixes were rather easy to implement in the code, they are all either relieving or quite helpful, and so I figured I would highlight them here rather as some part of a more minor section in this post.

Minor fixes and improvements

As usual, there were several more minor completed fixes and improvements that have taken place, actually more than usual in the past 2 weeks. The closer we get to release, the more larger systems will be complete and the more time I will be spending on things like this.

Most of the minor fixes this time involved some broken stuff with older staff UI, for example a master could not create a new directory in the root of the scripts folder. This was very rarely an issue, but it’s still nice to have resolved. There were also a few minor improvements to the logging system. For worldmakers, there have been more functions shared with the new scripting engine, in particular it came to my attention way later than it should have that worldmakers couldn’t check if a player was a newbie or not using the new scripting system, thus slowing down script conversions. This was the same with the player’s created timestamp and the server’s global temperature. These were probably reported to me sometime in the past, but likely while I was busily distracted coding something else and thus did not mentally register the reports properly. Also there is a dialog worldmakers are supposed to be able to use to add new world map table locations earlier, but it has been broken until only about a week ago. It wasn’t very high priority to fix since there was an alternate method (using slash commands) to add such locations if needed, but regardless though it’s good that worldmakers can finally add and edit locations directly from the world map table again. There were other very minor improvements for worldmakers, but those are the most major. For scribes, there has been a panel for quite a while now that would let one set the help index without needing to use a slash command, but it didn’t work correctly and fixing it was low priority since the slash command still worked. That has now been resolved and no slash commands are required what so ever for staff to manage the documentation anymore.

A couple other things include fixing a bug with creating characters that has existed since the SQLite conversion where their ID wasn’t being properly stored in the random access dictionary on character creation, causing any system that accessed characters by ID to fail for newly created characters until the server was restarted. Also there is a button in the view help topic dialog that opens the current topic in the web browser, it was broken though and that is no longer the case. So indeed most of the more minor improvements over the last couple of weeks have had to do with stuff only staff have access to, but first it’s important to remember that the more staff user interface we create the easier it is for staff to create new content for everyone and provide a fun gaming experience, and secondly this only means that the rest of the game is getting that much more stable since it’s getting harder for beta members in general to find bugs!

Tasks in progress

Meanwhile, there are a few tasks that are very actively being attended to. None of these are complete yet, but there is active progress happening in all of the following areas.

Documentation

I have already made most of the rules modifications needed now, and I have started doing some worldmaker documentation actually today. In edition, one of the worldmakers who is also a scribe who already understand many of the existing systems has volunteered to help me with this documentation, and they have informed me that they will send along some of what they have written in the next few days. For the main game’s documentation, we’ve added a few more scribes to the team to help get the public help articles ready. There was significant progress made there for a while early on, but up until now most of our scribes actually had that rank as a secondary position and were more focused on worldmaking aside from at the beginning when the system was added. The result is that we do have quite a bit of documentation written, however we decided that a few dedicated scribes pulled from the pool of applicants that seemed they’d be good for the job would be needed to finish the docs in time. They’ve already started work!

Tracking system

I’ve made much progress on polishing up the tracking system as I’ve mentioned in previous posts. There is still a bit for me to finish up, certainly it’s taking me a bit longer than the one day of dedicated coding I thought it would (do in large part to getting distracted fixing other stuff and needing to brainstorm more than I originally thought I would), but overall I’m satisfied with the progress I made early last week in this regard and look forward to beta members testing out all of the improvements almost certainly before the next blog post where I can then provide an update!

RTig to angelscript conversions

Of course, there has been more progress in regards to script conversions and more is still ongoing. Things are dragging a little here because for example we actually want to change some of the items as we’re converting them, cooking with spits being the largest example. However the number of remaining scripts continues to dwindle and this massive task continues to inch ever closer to being complete! Especially since many of the remaining scripts aside for a few exceptions are for more rarely used items and are also very tiny and repetitive, what I expect will happen is that not too much more progress will be made in this area until shortly before release, where I then suspect that the remaining scripts will be very quickly converted by quite a few worldmakers at a time in a couple of several hour long binge sessions. I’m hoping we can speed up the passive progress in this area a little over the next couple of weeks (for example by finishing up the worldmaker documentation) so that we have less to do during such final binge sessions, though regardless overall I’m not even remotely worried and think that this will all get done in time no problem however it happens.

tracking scripts and transfering them to main

This task has been mentioned before as the libgit2 integration, it’s the ability for worldmakers to transfer their work to the main server after creating something on the beta server. Though I haven’t started messing with libgit2 yet, what I did do is make it so that the shared scripting module can compile to bytecode for easy transferring to the main server by just copying a single blob of compiled script data. There is still quite a bit to do in this area, but I’m putting this up here instead of in the todo list below as with the ability to compile the scripts into bytecode now being a thing, I have officially started this process and it will likely be one of my main focuses over the next 2 weeks.

Tasks remaining

Now for the major items left in the todo list. I’ll avoid repetitively mentioning the tasks in progress above, as they are already well on their way to being complete.

We need to still spend time dealing with minor pathfinder issues. Though the pathfinder now works amazingly compared to before, we still need to fix that old previously mentioned bug dealing with npcs getting stuck at waterlines, and it would also be good if we can get npcs to stop causing some minor lag when they try calculating a path up a tree. As mentioned previously, backups is a huge one that I will attend to as soon as the game itself isn’t taking so much of my energy and focus. Though it’s not as important, the same goes for the stw website touchup. I also need to still debug the player watching system, though I’m seriously hoping that this will only take a few hours once I actually sit down and attend to it. Finally the introduction of the bug and suggestion tickets a couple of weeks ago has lead to a slew of more minor bug reports that need attending to, for example apparently drawing items too quickly causes the drawing to fail, we need to make it so you can drop a pile of items on a tile to change that tile (for example 25 wood to make a wood tile) again which won’t be that difficult, a couple item spawning issues etc. Overall not too many so far and I don’t think they’ll be that complicated to deal with. A couple other very minor things include converting a couple of really old travelpoints for more complicated maps to the new system (this one has been neglected for ages do to waiting for needed worldmaker infrastructure to be complete first), fixing offline notifications for deaths caused by dfcs, and improving our computer fingerprinting system as it is currently unreliable and causes the game settings data for some beta members to get corrupt.

Well guys, this section is getting harder and harder to write in terms of trying to think of content (I even asked some team members this time), and this is the one section where that is a very very good thing! We just need to finish the stuff mentioned in this section, the in progress tasks mentioned above and then take that final week to look things over/do final testing/get things ready, and then I’m pretty sure that sans anything I’m forgetting to write about right now which hopefully isn’t much, it’s finally going to be that time!

That’s all I’ve got for this one

Generally, I’d say these last 2 weeks have gone very well, a lot has gotten done in several random areas from documentation to worldmaker scripting to user interface and more! We are getting more and more excited as major tasks get completed and as the update gets closer, and can’t wait to share all of our work with everybody. As always, we want to thank every one of you for your patients and support during this time, it is extremely appreciated and helpful to know that we have a significant audience of players eagerly waiting to try out all we’ve done over the last year and more! As for me, I’m going to go enjoy my Easter weekend before I really get back to work on Monday, and I hope everyone else does as well! We’re getting seriously close I think you guys. Anyway I’m about completely out of things to say right now, so we’ll leave it here for now and I’ll see you all again in 2 weeks with more news. Thanks again everybody and to those who celebrate it, happy Easter!