Survive the Wild early March update

Hello everyone. It’s been a couple of weeks, so it’s time for another official update about the status of Survive the Wild. I’m going to try to post these either once every approximately 2 weeks, or whenever something particularly significant happens. We’ve had some awesome successes, a couple of setbacks and a more thorough todo list. Though some of it gets a little technical this time, I will describe all of it as it stands in this post.

Pathfinder improvements

This is something that I actually failed to mention in the last post, and I apologize for that. One of the things that has been holding us back for a while is the inefficiency of our pathfinder usage. The pathfinder is the component that scans the game map and generates intelligent paths for npcs to travel when going from destination to destination. For a little while after the July audio demo, the pathfinder worked but was inefficient. Then I tried to make it more efficient and broke it beyond much use. Having a lot of things to focus on I couldn’t always attend to it, so it remained broken for a couple of months while the Angelscript and Linux conversions were going on, leaving worldmakers to focus on script conversions and map/item/object work. A few months ago I improved the pathfinder to a point where npcs could at least spawn again, but not in large groups, or not for very long.

Well, I’m glad to say that we have finally resolved a large majority of our pathfinder implementation issues! Though there are still a few hickups from time to time, I have a little lag todo list, and once all items in it are implemented, we’re hoping for as close to perfect stability as we can manage. We’ve been running an extended test over the weekend, and are happy with the results thus far. We can do this while working because the server does not need to be restarted for scripting changes, and I’ve been doing other things which I’ll talk about in this post. So far though, the server has been running solidly for 4 days, 13 hours, 31 minutes, and 6 seconds. there are 1827 objects, 5509 items, 186 npcs, and 313 maps. Right now 83 of these npcs are moving while the rest are idling. Most of the idle npcs are resting, the few others are unfortunately stuck at waterlines. We have this bug right now where an npc will occasionally enter the water outside of their spawning zone and get hopelessly stuck. One npc will exhibit this behavior, and eventually die due to lack of food. Then another will find the exact same broken area because of the edible corpse left behind by the broken one, and it will get stuck too. This cycle continues until a giant pile of corpses and skeletons are all on one tile. We will of course attend to this, and once we do, a larger percentage of npcs will be moving around at a time. Though there are still a couple of things to resolve in regards to pathfinding, we are finally nearing a state where the lag caused by the pathfinder doesn’t severely inhibit the playability of the game and hope for that lag to reduce far more in the coming days! Of course I’ll provide another update on that in the next post.

Unexpected voice chat improvements

Sometimes I have to experience the problem to fix it. I recently moved back in with my parents, who have switched to using a cellular modem for their internet connection. This combined with any small amount of game lag would make it so that my own voice chats would be very breaky and jittery. I haven’t been very happy with this cellular internet connection, but since the debug data was available to me, I decided to take an hour and tackle the problem. The result is that voice chat is unexpectedly much more stable for everybody, usually even when the game is laggy! I just wanted to mention it quick as it’s a nice little unexpected bonus that I figured everyone would appreciate. Lag has decreased and stability has improved in almost all areas of the game in this version for an uncountable number of reasons, I’m glad we can specifically add voice chat improvements to that list!

Status on SQLite data conversion

I’m happy to say that I’ve successfully written scripts to convert the character data, the account data, and the account history to SQLite, and they have been tested on beta’s character database. They work great! Only a few minor modifications will be required to make all of these scripts work on main’s database. I’ll run them on it soon and have it prepared, but this will take place very near the public beta launch right as we’re getting the libgit2 integration mentioned in the previous post working, just encase any changes to the database structure take place before that time. This way, I don’t have to keep modifying the not yet live main database every time I change the internal structure, but can instead prepare it once near the release. I still need to write a script to convert the group chats data, but we’re nearly done with all this SQL conversion stuff! Support tickets are also theoretically converted, but we can’t test them yet. More on that below.

Status on the one large remaining admin UI panel

As mentioned in the previous post, I still have one more massive administration dialog to work on. This is the dialog that lets Admins view and modify anything necessary about characters, instead of using a bunch of legacy slash commands. We can change what controls appear in the dialog for each rank on the fly, for example a moderator may be able to restrict a person’s chatting, but the controls that allow them to view a player’s in-game location can be hidden. This is a very very complicated dialog with loads of controls. So complicated in fact that it resulted in a hiccup that lead to a little tangent. Basically there are several lists in this dialog, such as the one for restrictions. Rather than cluttering up the dialog even further with an add restriction, edit restriction and delete restriction or similar button set after each list (there are like 8 of them), the best way is if an admin can press ctrl+n in the restrictions list to add a new restriction, enter on a restriction to change it, or ctrl+delete on a restriction to remove it. I want this to be the same for all lists from quests to unlocked locations etc. The problem was that the remote dialog layer had no support for such shortcuts at all. So I had to take a break from working on the dialog itself to go add this support to the remote dialog layer in general. I have finished that however, and have made more progress on the dialog since then. It hasn’t been completed yet as there have been some distractions, such as the Angelscript bytecode loading issue described below. It’s getting there, however. I mean hey, at least now we can easily add any shortcut to any remote dialog, which in the future I imagine will help the UI along for players as well.

Miscellaneous improvements

There have of course been some more minor improvements over the last two weeks that aren’t quite deserving of their own section. The most important of these is that it is no longer possible to double press buttons in dialogs. If the server was laggy, or if a player simply pressed enter two fast, they could accidentally perform any number of transactions multiple times, and I don’t even want to think about how many bugs it was causing. For example we previously had issues regarding renaming and deleting characters, but only occasionally. I sometimes wonder now if some of those issues were because the rename button was clicked twice very quickly in a row when the server only expected one valid renaming or deletion dialog submission, though even if this wasn’t the problem the old file based character system (prior to SQLite) certainly was. Now renaming or deleting a character simply involves changing a field in a database, not renaming files and modifying the linked account data to represent it as before. Similar in severity but in different ways, a user could also double click a purchase button, thus spending twice or more the amount of credits or points they intended! Someone could give an item twice if they had that quantity, etc. Needless to say it’s a good thing that this was resolved, now the dialog locks up until a response from the server is issued so that buttons cannot be pressed multiple times before the server replies to the first button press at least once.

I’ve also finally given worldmakers and scripters the ability to alter more player properties last week, basically a scripter can now alter the jumpheight, autowalk speed, tile tripping limits, etc for a player. This could have any number of effects in the future, from an over large pair of snowshoes that make it easier to walk in snow while making it more likely you’ll trip on other surfaces, to arenas where you can pretend to be a superhero with speed, jumpheight and other stats that could never be acceptable in the main game. I’ve just added the feature, it will be fun to see what worldmakers do with it!

Another minor useful edition is that last week I was able to successfully link with a cross platform system information library. I’ve had my eyes on this library for a month or so now, and have finally wrapped it in NVGT. This is nice because now the server can detect, for example, if the server is running out of disk space or ram, and can now automatically react accordingly. I can also finally check how much memory the server process is taking directly from within the game, so no more having to ssh into my server and request a systemctl status on the server’s service for that simple bit of information, yay!

Other than that, most other minor things I’ve done are unfortunately too small to readily remember and thus list. Due to npc fixes and trying to test support tickets actually, we did improve our exception logging significantly. I have so much more info as a developer now, as whenever there is a minor exception either in a shared item/object script or in the main game module, all call stack information is now logged somewhere instead of just the exception itself. Worldmakers and scripters can access these exceptions, so they are also able to debug their own scripts much more easily now. Similar improvements to the logging facilities now allow us to find most problematic npcs very quickly and go attend to them. Well, at present that’s all I can think of in regards to minor improvements since the last post.

This sql conversion has caused a previously minor Angelscript bug to become more critical

We’ve had a long standing issue with Angelscript pretty much since STW first ran in NVGT in March 2022. It’s hard to explain it in detail without getting overly technical, but basically I found that the first time I tried compiling Survive the Wild instead of running it from source, the Work-In-Progress (development version) of Angelscript that I was using was unable to load the compiled bytecode of the game. I was only using a WIP version in the first place, because the previous public version before that had a backwards compatibility issue with ternary operators. Basically because I use ternary operators all over my code, I was unable to compile or even run stw using version 2.35.1 of Angelscript, which was the latest at the time. Though I tried various versions, the end result was me spending hours rolling back the Angelscript commit history to find a WIP version new enough to have fixed ternary operators, but not so new that the bytecode loading errors were introduced. We’ve been using this WIP Angelscript version from January 2022 for months now.

And then, I wrote that wrapper around SQLite that I mentioned in the previous post. Everything seemed fine. I converted and tested characters/accounts, everything still seemed fine. Then I converted tickets and tried to test them, and that’s where we ran into a problem. Unfortunately, in that old WIP Angelscript version I was using, the operator overload that lets me convert a class back into a string is somewhat faulty. It seems that this bug was likely discovered and fixed in a later WIP Angelscript version, which I cannot upgrade to because it will produce a bytecode loading error if I try to run the compiled game.

So, left with the options of hacking around with older angelscript versions/dirtying my codebase to try to get tickets working, or contacting the Angelscript developer to actually get the bytecode loading bug fixed, I chose the latter. As of yesterday I’ve already contacted the developer, and I’m happy to say he’s already responded! Therefor, I spent most of today gathering more debug information based on his instructions, and have sent him an update. I’m hopeful that we’ll be able to upgrade Angelscript and test the conversion of support tickets to SQLite soon! I’m not even sure that this character administration dialog will work either until this is fixed, as many values from the character are casted to strings to show them in the dialog. As I said earlier I may be able to negate this by dirtying my codebase, explicitly calling string constructors all over the place etc, but would obviously rather not do that for several reasons. Not to mention, I learned a valuable lesson about reporting Angelscript bugs as soon as I find them. Due to my failure to contact the Angelscript developer before now, this obscure bytecode loading issue actually made it into the public Angelscript 2.36.0 release in late September of 2022 even though I’d known about the issue since March of that year, stopping me from using that as well. Therefor taking a couple extra days to properly resolve this with the Angelscript developer helps not only me/the game because it will allow us to use the latest, fastest Angelscript, but it will also potentially help every Angelscript user who may also run into the bytecode loading bug in the future. Anyway, today I was able to distill this bytecode loading issue into a tiny tiny sample of code and have sent that off to the Angelscript developer for analysis, you’ll hear the end result of that hopefully by the next blog post at the latest!

So what’s left to get done?

Please keep in mind that our todo list expands and shrinks as we discover issues, fix them, and make general progress towards this release. Not only that, but it’s difficult for me to remember everything that needs doing all at once, as you may have seen I managed to completely forget about the pathfinder in the previous post. Therefor if I forget something major here, I do apologize, I’ll of course try to be more thorough than last time.

Though the server is mostly stable now, we still have some pathfinding related issues to resolve as mentioned above. The one I’m most excited about getting done is the calculate_distance_desperation function. Right now, when an npc tries finding food, it fetches a list of nearby items, and tries pathfinding to the one with the closest straight line distance from the npc. This has been leading to several problems, for example an npc will cross a stream to grab a food item that’s 20 tiles away, while ignoring the other, much more easily accessible food item that’s on the same side of the stream but is 30 tiles away. Similarly an npc will climb a tree to grab an apple just because it’s a few tiles closer in a direct straight line, rather than going for the much more easily accessible apple that fell from the tree onto the ground not much further away. We only want to calculate complicated paths when absolutely necessary, both because doing so too often makes an npc look a little dumb, and also because the more complicated the path, the more aggressive the lag can get. See, water has a higher desperation factor than grass. This means that usually npcs will avoid the water unless they need it or are meant for it. So when an npc tries calculating a path across the stream, it will scan literally every tile on it’s current side of the stream first, up to the maximum pathfinder search range. It does this because do to the very nature of the pathfinder, it must calculate the path with the least cost. This means the pathfinder must assure itself that there is no easier way to cross the stream than just crossing the stream. For example the pathfinder is looking for a tunnel under the stream, a tree with branches long enough to cross it, a gap in the water where it can walk on land to cross etc. A great example of this is the clay on the mainland, which is surrounded on 2 sides by stream, one side by swamp and the other side by beach and then ocean. So if an npc that is in the clay area tries to cross the stream, it doesn’t always actually cross, but instead may go all the way south to the beach followed by moving east along the sand, before heading north to the intended destination on the northeast side of the stream. This sort of thing can happen with all kinds of other tile types and eventually can get very costly, so we therefor must minimize such complicated paths as much as possible. To do this, I intend to write a function that does calculate distance between one object and another , but not in a straight line. This function will instead sum up all desperation factors of all tiles between the first and the second object in a straight line. The result should be that if a straight line path crosses tiles with a high desperation factor, the object’s distance from the npc when calculated with this function will be much higher than an object that actually has a greater straight line distance, but with lower desperation tiles between itself and the npc. After this function is written, we should be able to select the easiest food item to go retrieve rather than the theoretical closest. There are other smaller improvements we need to make with the pathfinder, such as npcs occasionally getting stuck as mentioned above. Just talking about one though in this much detail is probably boring for some readers, so I’ll stop with that one and just say for the record that I’m extremely excited to clear out this pathfinder related lag todo list and see how stable the server gets as a result, we’re already very much getting there!

Then there is of course finishing this final UI panel which as I mentioned was already started. We have some smaller UI related things to solve, such as some editions for admins that will be so easy they aren’t worth mentioning here, and some things to fix in a couple of menus that I am not sure I’m ready to spoil right now. Considering that I’m having a hard time thinking of any other UI improvements that need to be made, I think we’re finally nearing the end of that stuff as well!

I still must attend to this libgit2 integration mentioned in the previous post, this is the thing that will let worldmakers transfer their work from the beta/development server to the public game. Allowing scripters and worldmakers to script directly on the public game is far too dangerous for several reasons, so getting this one working before the release is a must.

I also have a bunch of English writing to do aside from the code. I need to update a few things in the rules document, and though it’s less critical, I should probably touch up the stw website a bit before it goes live. While this may seem like a temporary waste of time for experienced players, it’s important to remember that new players will find the game through this website as well. Currently this website only has functionality, it does not yet have large blobs of text that, for example, explain what Survive the Wild is, the features, and other various information. We have a credits page, but so far it only lists the automatically generated music credits, not staff credits yet. Though theoretically not required before the public beta release, it’s probably a good idea to get this website touchup out of the way. I’m pretty sure that public beta will present an initially sizable bug list which will need resolving, and this would distract me from being able to fix the website sometime shortly after the game goes live. It seems prudent to do it ahead of time as I do not know when the first relatively calm period will take place after public beta releases.

The RTig to Angelscript conversion is still going on. As I’ve been focused mostly on the hardcode, I have not been monitoring this closely enough to provide detailed statistics on what’s left there, but for sure by the next blog post I’ll make sure to give more details on that if it’s not already done by then.

I also must still take a day or 2 and attend to the tracking system as mentioned in the previous blog post. It’s already immeasurably better than the old system, but it still needs polishing before it’s completely ready for use without being a bit annoying to operate. It’s one thing I haven’t started touching yet since I first talked about it

Since I haven’t been able to test tickets yet, I still have some work to do to make them have categories such as bug and suggestion. The database has a type field for each ticket now, but really I still have to do most of the grunt work involved to make each type of ticket show up in the correct menu, make each ticket type have different states (such as critical or duplicate for bug or awaiting communication/resolved for general tickets), and a couple other things related to the system. I really wish I’d thought of this when the ticket system was first being coded, however we didn’t come up with the idea to make bug reports and suggestions a type of ticket until sometime after the system had been complete. I’m not worried, once tickets (converted to SQLite) have been confirmed functional it should only be a few hours of dedicated work to get this done.

One thing that’s easy to forget about at least right now with other stuff to attend to is that I need to fix our PayPal integration. Stripe works beautifully, but in mid 2022 when I created the new website, I was having some minor nightmares trying to deal with PayPal’s sandbox account pages. I think I got it working with PayPal’s antiquated and legacy IPN (Instant Payment Notification) system. However this system is, if not officially considered deprecated, very old and it can sometimes take ages for a payment notification to reach my server, causing players to wonder why they aren’t seeing their credits. An admin would then give the player replacement credits considering the transaction notification to have somehow gone wrong, only to see the actual transaction notification appear an hour later thus giving the player double credits. PayPal has created a much more efficient system called webhooks, which uses much more standard technology, is much more instant, and is much newer and therefor more supported. As such, I intend to replace the old IPN system with the new webhooks one before the store goes live. However, since the store will be offline during public beta anyway, I may be able to get away with getting this one resolved after the first release rather than before. PayPal has luckily updated their sandbox account pages. Where as before they were badly next to unusable with a screen reader, I’m happy to say that they now appear beautifully accessible and so I should have no more issues operating them!

We are planning to change how deleting characters works. In the next version, assuming it has no linked characters, you can now finally mark your account for deletion after 30 days of no use. We want to implement this for characters as well, where instead of instantly deleting a character, you mark the character for deletion, which then takes place 30 days later assuming you have not logged into the character since you marked it to be deleted. This however isn’t particularly critical and we may save it for after the first public beta release.

Finally, we also still intend to take this extra week once we’re all done with everything to look over our work and to refine some content. This was mentioned in the previous blog post.

At the moment, that’s all I can think of. Though I checked our todo list, some tasks do not always end up getting written down in it either because they are very small, very transient or show themselves repetitively enough that we don’t need to write them down to remember them when it matters. I think we’ve been pretty thorough with it, but I want to apologize ahead of time if I’m forgetting something else we need to do that suddenly pops up in the next blog post, as has happened with this one. Hopefully though this section has listed the vast majority of tasks we must complete before releasing the update, and even some short-term items that we are considering saving until after the first release.

And so, progress continues

Well folks, I think that’s all of the updates I can think of at this time. As you can see progress continues day by day, and we are as excited to show what we’ve been working on to all of you as players are to enjoy it! As always, thank you all for your patients, kindness, support and understanding during this unfortunate downtime. Our goal remains the same, to get this update out to everyone as soon as we are able to logically do so!

Huh what do you know, at least my blog posts are getting… IDK, somewhat more positive? What can I say, progress, at least progress!