Feature Specification: Persistent Storage

[Ronan]

I just realized we don't have a persistant storage system defined... I am very tired at the moment, so I'll be brief, but the following is what this proposed system has over currently-used systems:

-Supports pseudo-transactions. If there is any error writing to the DB, or if the server crashes, the whole write does not take effect. If the server didn't crash and the write just didn't succeed, it will try again.
-Similarly, if there is any error reading the DB, it can default back to its previous contents. Maybe have it re-try the read first?
-All PCs who disturbed the chest's inventory are saved as it closes (assuming we have a better vault set up in NWN2, and the unclosing-chest bug is solved or absent).
-Data can be read off the chest for use in logging and DM tools.
-Each pChest will get its own seperate DB (actually two, to use pseudo-transactions). So if one file is corrupted due to a hard crash or the like, no other chests will be effected.

If time permits, we could include tools for a DM to find the user(s) of a pChest, and find the pChest(s) owned by a specific PC. I think its more important to get the core functionality down first though, since as long as everyone is using this system we can always seemlessly update later.

Ties with other systems:
acr_tools_i (general tools)
acr_sv_persist_i (persistant variables local to the server)
acr_storageobj_i (organizing of in-game local variables)
acr_array_i (array support, for local variables, server variables, and pc-persistant variables)
acr_debug_i (debugging message tools)

Very incomplete code follows, comments, suggestions? These are part of the core, required scripts IMO, and should be done as well as possible.

[Ronan]

Code: Select all

////////////////////////////////////////////////////////////////////////////////
//
//  System Name : ALFA Core Rules
//     Filename : acr_pers_stor_i
//      Version : 0.1
//         Date : 5/16/06
//       Author : Ronan
//
//  Local Variable Prefix = ACR_PSO
//
//
//  Dependencies external of nwscript: None
//
//  Description
//  This script handles all persistant storage used by PCs to store items and
//  gold. All persistant storage should use these scripts, and none other. It
//  also logs and evaluates the contents of persistant storage, and links
//  to the last PCs which have used it.
//
//  Revision History
////////////////////////////////////////////////////////////////////////////////

////////////////////////////////////////////////////////////////////////////////
// Includes ////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////

#include "acr_sv_persist_i"
#include "acr_array_i"
#include "acr_tools_i"
#include "acr_debug_i"

////////////////////////////////////////////////////////////////////////////////
// Constants ///////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////

// The number of different PCs the system tracks as having accessed each
// persistant storage object.
const int ACR_PSO_RECORD_USERS = 5;

// Should the PCs who disturbed the storage object's inventory be saved to the
// vault when the object is saved?
const int _ACR_PSO_EXPORT_PC_ONSAVE = 1;

// Should the DMs who disturbed the storage object's inventory be saved to the
// vault when the object is saved?
const int _ACR_PSO_EXPORT_DM_ONSAVE = 0;

// Constants representing the desired behavior of the storage if it is
// destroyed.
const int _ACR_PSO_ON_DEATH_HIDE_ITEMS = 0;
const int _ACR_PSO_ON_DEATH_DROP_ITEMS = 1;
// ADD ME!! Spawn another pStorage object on death, so what is not looted is
// still saved !!

// The number of backups to maintain, in case of data loss.
const int  _ACR_PSO_NUMBER_OF_BACKUPS = 3;

// The local variable which contains a string which uniquely identifies a
// persistant storage object.
const string _ACR_PSO_DATABASE_NAME_LS = "ACR_PSO_NAME";

// The variable name to retrieve the number of items in a storage object.
const string _ACR_PSO_NUMBER_ITEMS_VARIABLE = "NUM";

// The local integer defining the desired behavior when the pChest is destroyed
const string _ACR_PSO_ON_DEATH_BEHAVIOR_LI = "ACR_PSO_ONDEATH";

// Local integer signifying the object was disturbed since its been opened.
const string _ACR_PSO_DISTURBED = "ACR_PSO_DISTURBED";

////////////////////////////////////////////////////////////////////////////////
// Structures //////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////

struct strStorageTransactionInfo {
    int nNetGoldToPC;
    string sPCName;
    string sPlayerName;
    int nNumberOfTransactions;
};

////////////////////////////////////////////////////////////////////////////////
// Global Variables ////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////

////////////////////////////////////////////////////////////////////////////////
// Function Prototypes /////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////

// Initialize the persistant stoage system.
void InitializePersistantStorage();

// Handles the OnOpen event of a persistant storage object.
void PersistantStorageOnOpen(object oStorage);

// Handles the OnClose event of a persistant storage object.
// This function saves the contents of the chest and all PCs who have disturbed
// its inventory.
void PersistantStorageOnClose(object oStorage);

// Handles the OnDisturb event of a persistant storage object.
void PersistantStorageOnDisturb(object oStorage, object oPC);

// Handles the OnDeath event of a persistant storage object.
void PersistantStorageOnDeath(object oStorage);

// Returns the cumulative value of all the items in this storage.
int GetPersistantStorageValue(object oStorage);

// Returns the number of items in this storage object, or 0 if oObject is not
// a persistant storage object.
int GetNumberOfItemsInStorage(object oObject);

// Returns the nIndex item in oObject's persistant storage, or OBJECT_INVALID
// if oObject is not a persistant storage object, or if there is no item at
// nIndex.
object GetItemInPersistantStorage(object oObject, int nIndex);

// Returns 1 if oObject is a persistant storage object, and 0 otherwise.
int GetIsObjectPersistantStorage(object oObject);

// Loads the inventory of persistant storage object oObject.
// If oTarget is specified, it loads the contents into another object with an
// inventory. Whatever the target, the inventory of it is DESTROYED, and
// replaced with the objects stored in the database.
// Returns the number of items loaded.
int _LoadStorageObjectContents(object oObject, object oTarget = OBJECT_INVALID);

// Writes the contents of oObject to the server's local persistant storage.
// It checks for errors as it goes, and if an error is detected, the entire
// write is aborted, and the storage object reverts to the old version.
int _WriteStorageObjectContents(object oObject);

// Return's 1 if oObject's storage contents have been loaded, or 0 if not.
// Also returns 0 if oObject is not a persistant storage object.
int _GetAreContentsLoaded(object oObject);

// Returns the number of PCs who have disturbed oObject's inventory since the
// last time it was closed.
int _GetNumberOfDisturbers(object oObject);

// Adds a PC or DM to the list of characters who have disturbed this object's
// inventory since the last time it was closed.
int _PushDisturbers(object oObject, object oPC);

// Clears the list of PCs who have disturbed this storage object's inventory.
// Generally called OnClose.
void _ClearDisturbers(object oObject);

// Returns oObject's nIndex disturber, or OBJECT_INVALID on an error.
object _GetDisturber(object oObject, int nIndex);

////////////////////////////////////////////////////////////////////////////////
// Function Definitions ////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////

void InitializePersistantStorage() {
    _persistantStorageDebugId = CreateDebugSystem("Persistant storage: ", DEBUG_TARGET_NONE, DEBUG_TARGET_LOG, DEBUG_TARGET_LOG);
}

int _GetIsObjectPersistantStorage(object oObject) {
    return GetLocalString(oObject, _ACR_PSO_DATABASE_NAME_LS) != "";
}

int _LoadStorageObjectContents(object oObject, object oTarget = OBJECT_INVALID) {
}

void PersistantStorageOnOpen(object oStorage) {
    if(!_GetAreContentsLoaded(oStorage)) {
        _LoadStorageObjectContents(oStorage);
    }
}

void PersistantStorageOnClose(object oStorage) {

    // If the chest's contents haven't been altered, we do nothing.
    if(!GetLocalInt(oStorage, _ACR_PSO_DISTURBED)) {
        return;
    }

    // Find every exportable character who has disturbed this object, and export
    // them.
    int nDisturbers = _GetNumberOfDisturbers();
    int i;
    for(i=0; i<nDisturbers; i++) {
        object oPC = _GetDisturber(oObject, nDisturbers);
        ExportSingleCharacter(oPC);
        // Update oStorage's list of transactions?!?
    }
    _WriteStorageObjectContents(oStorage);
}

void PersistantStorageOnDisturb(object oStorage, object oPC) {
    _PushDisturbers(oStorage, oDisturber);
    // Record the transaction?!?
    SetLocalInt(oStorage, _ACR_PSO_DISTURBED, 1);
}

void PersistantStorageOnDeath(object oStorage) {
// ADD ME!! Spawn another pStorage object on death, so what is not looted is
// still saved at reset !!
    int nOnDeath = GetLocalInt(oStorage, _ACR_PSO_ON_DEATH_BEHAVIOR_LI);
    if(nOnDeath == _ACR_PSO_ON_DEATH_HIDE_ITEMS) {
        DestroyInventory(oStorage);
    } else if(nOnDeath == _ACR_PSO_ON_DEATH_DROP_ITEMS) {
        if(!_GetAreContentsLoaded(oStorage)) {
            _LoadStorageObjectContents(oStorage);
        }
    }
}

[Baalster]

Splendid idea. And another thing we should make sure is in the module, are flagsets - or bitwise operators. They are marvellous things for keeping track of large amount of stuff. I used it for my explorer quests, which consisted of three levels with up to 8 locations to find. And all those 24 bits left 8 bits over for a progress counter, and was stored on one interger variable.

Baalster

[ç i p h é r]

The current pChest implementation suffered from three things IMO:

1. not saving character inventories when storing items
2. not purging items from the database that have been removed
3. not having independent databases for each chest or PC

On the 3rd point, if using the NWN db, we'll need to be certain we have uniqueness in database names to avoid access issues. If we can auto generate names (perhaps use location?), that would mean builders need not do anything more than place the chests.

On the point about backups, are you saying that the application should backup database files in real time? My concern there is that real time backups will introduce delays as writes tend to be fairly time consuming. Depending on the system, maybe database backups could be scheduled in the background, if it's essential.

Oh and one last point. I think the NWN database will probably be easier to use as an "item vault" than a generic SQL database, if it's the same as the Bioware DB. It stores all item data, including variables, when stored. Since item vaults will be location specific, we don't necessarily need to store the data centrally, but it would help if we had a LOG of what was stored in them centrally for administrative purposes, like wealth calculation.

P.S. I recall some issues with storing gold, but I can't recall what they were off the top of my head at the moment. But, gold should be something PCs can store in a pChest, right?

[Ronan]

The current pChest implementation suffered from three things IMO:

1. not saving character inventories when storing items
2. not purging items from the database that have been removed
3. not having independent databases for each chest or PC

Agreed mostly, and I think this system covers all those points. But I would also add not being able to recover from a failed write (hard crash, power outage, whatever) which my system should be able to do.

On the 3rd point, if using the NWN db, we'll need to be certain we have uniqueness in database names to avoid access issues. If we can auto generate names (perhaps use location?), that would mean builders need not do anything more than place the chests.

I think the db names need to stay builder-defined. I don't know any other way of letting them be totally movable and recoverable (after a delete, etc).

On the point about backups, are you saying that the application should backup database files in real time? My concern there is that real time backups will introduce delays as writes tend to be fairly time consuming. Depending on the system, maybe database backups could be scheduled in the background, if it's essential.

Mechanically, it woud work like this: Each chest would have two databases, and a variable which marked which database was "current". When writing, the chest would write to the older database. If the write completed without any errors (checked in real-time, though we could stop this if we wished to trust the return value of BW's StoreCampaignObject()), this other variable would be changed to point to the current database. If there was an error reading anything, it would be changed to point back to the later. I think this emulates a transaction in a modern SQL database, where changes are only commited after all the writes in a transaction are completed successfully.

Oh and one last point. I think the NWN database will probably be easier to use as an "item vault" than a generic SQL database, if it's the same as the Bioware DB. It stores all item data, including variables, when stored. Since item vaults will be location specific, we don't necessarily need to store the data centrally, but it would help if we had a LOG of what was stored in them centrally for administrative purposes, like wealth calculation.

Ah, good point. I'll add hooks to the 1984 scripts, which we can fill in later as our logging options become more apparent.

P.S. I recall some issues with storing gold, but I can't recall what they were off the top of my head at the moment. But, gold should be something PCs can store in a pChest, right?

I think it doesn't flag OnDisturbed, in which case we need to count all the different PCs who flag OnClicked and not OnDisturbed, and check for a change in gp in the chest OnClose.

[ç i p h é r]

I think the db names need to stay builder-defined. I don't know any other way of letting them be totally movable and recoverable (after a delete, etc).

That's a good point. It's possible if db names were computed with values you could deduce from the toolset, but it's not fool proof. Say for instance the formula is area name + location coordinates. The area is obvious and the coordinates can be determined by using the Adjust Location popup menu option. The risk is forgetting to do that before you move (or remove) the chest....ooops. Though your odds of finding the chest are still fairly reasonable as the area name/tag would be in the db name as well. Conversely, if db name is builder defined and they adopt goofy naming conventions, the database may be even harder to locate and recover after a delete, though clearly they can be moved without risk.

Anyway, it's just a thought toward convenience. Perhaps we can give builders the choice and use builder defined db names if pChest TAG != NAME.