Serialization

Serialization is the process by which a collection of PhysX objects is stored in a persistent form outside the PhysX runtime, such as on disk. Deserialization is the reverse process, i.e. the loading of those objects into another instance of the PhysX runtime. PhysX 3 features two serialization APIs:

  • API-level serialization to RepX, a versioned XML data format.
  • Binary serialization, which serializes objects into a block of memory from which the PhysX runtime can later load them without allocation or copying.

Both serialization systems use their own meta data representation of PhysX data structures. The meta data used by RepX captures the PhysX objects at the API-level. It allows forward conversions of serialized data from earlier PhysX 3 versions to later ones. The binary serialization meta data captures the internal data structures of PhysX objects and can be used to convert binary representations between different platforms.

Note

cooking also generates a binary output stream. However the primary purpose of cooking is to translate from a user format to a format suitable for the SDK runtime, and so it is not considered a serialization mechanism. Loading a cooked mesh from a stream involves allocation and endian conversion, so is much less efficient than PhysX' binary serialization mechanism. See Shapes and Geometries for more details about cooking.

Serialization and Deserialization

Here is a brief overview on the main serialization types:

  • PxBase is the base class for the objects that can be serialized.
  • PxCollection is a collection of PxBase objects. Serializables are always serialized and deserialized as part of collections.
  • PxSerialObjectRef is a 64 bit type, which is used as a reference to a serialized object.
  • PxSerializationRegistry is a registry for serializable types inheriting from PxBase.
  • PxSerializer are needed for both binary and repx serialization.
  • PxSerializerDefaultAdapter are needed for both binary and repx serialization.
  • PxRepXSerializer is additionally needed for repx serialization, plus the corresponding registration function.

Serializables can be added to and removed from collections. A serializable may appear in multiple collections, but needs to be unique within a collection. Serializables can be given a name (PxSerialObjectRef) within a collection. It's ensured that the name is unique within a collection as well, however it's the applications responsibility to consistently name a serializable across different collections.

Here is an example of how to iterate over a collection, for instance to ensure the objects you intend to serialize have all been added to the collection. When doing so you can use PhysX' dynamic typing mechanism to classify the objects:

PxCollection* collection;
PxU32 size = collection->getNbObjects();
for(PxU32 i=0; i<size; i++)
{
    PxBase* object = collection->getObject(i);
    if(!object->is<PxActor>())
        continue;

    switch((PxConcreteType)object->getConcreteType())
    {
    case PxConcreteType::eRIGID_DYNAMIC:
    ...
    }
}

The following two sections describe how PhysX object graphs can be serialized and deserialized using the binary format or RepX. The first section shows how to deal with collections that describe complete object graphs. The second section shows how to deal with collections describing incomplete object graphs, i.e. collections that reference other collections.

Serializing Complete Object Graphs

This code snippet shows how to prepare a collection of PhysX objects for serialization (e.g. an actor, its shapes, and the materials and meshes they reference.):

PxPhysics* physics;                                         // The physics SDK object
PxRigidDynamic* dynamic = PxCreateDynamic(...);             // Create a rigid dynamic
PxSerializationRegistry* registry = PxSerialization::createSerializationRegistry(*physics);  //Create a serialization registry

PxCollection* collection = physics->createCollection();     // Create a collection
collection->add(*dynamic);                                  // Add it to the collection

PxSerialization::complete(*collection, *registry);          // Adds all serializables required to recreate the dynamic after deserialization

Alternatively to using PxSerialization::complete you could also add the serializables which are required for serialization manually. You would need to add all serializable objects the PxRigidDynamic requires and then recursively add all serializable objects required by the newly added objects. See definitions: Requires, Complete

By default PxSerialization::complete does not add joints to actors, since actors don't require their joints. Unless one adds joints manually to the collection they will be ignored on serialization and deserialization. The followJoint parameter can be used to change the behavior of PxSerialization::complete to implicitly add the requires relationship actors require their joints. This will cause entire actor-joint chains to be added to the collection.

When all the necessary objects have been added to a collection, create an implementation of the PxOutputStream interface, then serialize the collection:

PxPhysics* physics;                                       // The physics SDK object
PxColletion* collection;                                  // Complete collection without orphans
PxSerializationRegistry* registry;                        // Registry for Serializable types
PxOutputStream& outStream = ...;                          // Implemented by the application

// Serialize

// Binary
    PxSerialization::serializeCollectionToBinary(outStream, *collection, *registry);
//~Binary

// RepX
    PxSerialization::serializeCollectionToXml(outStream, *collection, *registry);
//~RepX

collection->release();
registry->release();

The following code shows how to deserialize a collection from a memory block or xml:

PxPhysics* physics;                                       // The physics SDK object
PxSerializationRegistry* registry;                        // Registry for Serializable types
PxCooking* cooking;                                       // Cooking library needed for instantiating objects by repx

// Deserialize

// Binary
    void* memory128 = ...;                                // A 128-byte aligned buffer previously loaded from disk by the user
    PxCollection* collection = PxSerialization::createCollectionFromBinary(memory128, *registry);
//~Binary

// RepX
    PxInputData& inputData = ...;                         // Implemented by the application
    PxCollection* collection = PxSerialization::createCollectionFromXml(inputData, *cooking, *registry);
//~RepX

To add all the objects to the scene and release the collection and registry:

PxScene* scene;                                           // The scene object
scene->addCollection(*collection);
collection->release();
registry->release();

In order to successfully serialize and deserialize a collection it needs to maintain certain properties. See Serializable. Whether these properties are met can be checked with PxSerialization::isSerializable(...).

Serializing Partial Object Graphs

The example above serializes a collection representing a complete object graphs. Another common use case is where a collection of actors and joints - say, a rag doll - will be deserialized multiple times, with each instance sharing the same materials and meshes. To achieve this, serialize two collections:

  • a collection of the materials and meshes that will be deserialized just once
  • a collection of actors and joints which will be copied and deserialized multiple times

The second of these will be a partial object graph: there will be some objects which are not serialized with the collection, but to which the collection will contain references. The application needs to do the following to serialize and deserialize a partial object graph:

  • Provide consistent reference names for all objects that are referenced across collections.
  • Provide the referenced collection with consistent reference names on serialization and deserialization

Here are examples of how the application can provide reference identities (PxSerialObjectRef) to express requirements of one collection to another. This can be done explicitly when adding the serializable with:

PxCollection* collection;
PxTriangleMesh* triMesh;
PxSerialObjectRef triMeshId = 1;                 // PX_SERIAL_OBJECT_REF_INVALID is a reserved value

collection->add(*triMesh, triMeshId);

Or set the reference after adding the serializable:

collection->add(*triMesh);
collection->addReference(*triMesh, triMeshId);

There is a helper function to name all serializables in a collection that don't have names yet:

PxSerialObjectRef baseId = 1;                           // PX_SERIAL_OBJECT_REF_INVALID is a reserved value
PxSerialization::createNames(*collection, baseId);      // Assignes incremental reference values to the collection objects

Already used reference values will be skipped by createNames, as well as serializables that already have reference names.

After providing correct reference names one needs to make sure all required objects have been added to the collection to be serialized, but without adding the objects that are intended to be referenced. The complete function in PxSerialization supports completing a collection relative to another collection:

PxCollection* incomplete;                                        // Collection to be completed
PxCollection* exceptFor;                                         // The collection, incomplete will depend on
PxSerializationRegistry* registry;                               // Registry for Serializable types
PxSerialization::complete(*incomplete, *registry, exceptFor);    // Completes incomplete, but ignores objects in exceptFor (and also their requirements)

Serialization example:

PxConvexMesh** convexes;             // An array of mNbConvexes convexes
PxRigidDynamic** actors;             // An array of mNbConvexes actors referencing the convexes

PxPhysics* physics;                  // The physics SDK object
PxSerializationRegistry* registry;   // Registry for Serializable types
PxOutputStream& convexStream;        // Output stream for the convex collection
PxOutputStream& actorStream;         // Output stream for the actor collection

PxCollection* convexCollection = physics->createCollection();
PxCollection* actorCollection = physics->createCollection();

// Add convexes to collection
for(PxU32 i=0;i<mNbConvexes;i++)
    convexCollection->add(*convexes[i]);

// Create reference names for the convexes, starting with 1
PxSerialization::createNames(*convexCollection, PxSerialObjectRef(1));

// Serialize the convexes along with their reference names

// Binary
   PxSerialization::serializeCollectionToBinary(convexStream, *convexCollection, *registry);
//~Binary

// RepX
   PxSerialization::serializeCollectionToXml(convexStream, *convexCollection, *registry);
//~RepX

// Add actors to other collection
for(PxU32 i=0;i<mNbActors;i++)
    actorCollection->add(*actors[i]);

// Add all required serializables except the convexes
PxSerialization::complete(*actorCollection, convexCollection, *registry);

// Serialize the actors with references to convexCollection

// Binary
    PxSerialization::serializeCollectionToBinary(actorStream, *actorCollection, *registry, convexCollection);
//~Binary

// RepX
    PxSerialization::serializeCollectionToXml(actorStream, *actorCollection, *registry, NULL, convexCollection);
//~RepX

// Release collections and registry
convexCollection->release();
actorCollection->release();
registry->release();

Deserialization example:

PxPhysics* physics;                         // The physics SDK object
PxSerializationRegistry* registry           // Registry for Serializable types
PxCooking* cooking;                         // Cooking library needed for instantiating objects by repX
PxScene* scene;                             // The scene into which the objects will be inserted

// Deserialize convexes along with their reference names (no external dependencies)

// Binary
    void* convexMemory128;                  // Aligned memory containing serialized convexes
    PxCollection* convexCollection = PxSerialization::createCollectionFromBinary(convexMemory128, *registry, NULL);
//~Binary

// RepX
    PxInputData& convexInputData = ...;     // Implemented by the application
    PxCollection* convexCollection = PxSerialization::createCollectionFromXml(convexInputData, *cooking, *registry, NULL);
//~RepX

// Deserialize actors referencing the convexCollection

// Binary
    void* actorMemory128;                   // Aligned memory containing serialized actors
    PxCollection* actorCollection = PxSerialization::createCollectionFromBinary(actorMemory128, *registry, convexCollection);
//~Binary

// RepX
    PxInputData& actorInputData = ...;      // Implemented by the application
    PxCollection* actorCollection = PxSerialization::createCollectionFromXml(actorInputData, *cooking, *registry, convexCollection);
//~RepX

// Release convex collection
convexCollection->release();

// Add actors to scene and release collection and registry
scene->addCollection(*actorCollection);
actorCollection->release();
registry->release();

The next example shows how to deal with situations where your serialized objects require objects that are not serialized and deserialized but created by other means:

PxPhysics* physics;                 // The physics SDK object
PxSerializationRegistry* registry;  // Registry for Serializable types
PxMaterial** materials;             // Created procedurally by application
PxRigidDynamic** actors;            // An array of mNbConvexes actors referencing the convexes
PxOutputStream& actorStream;        // Output stream for the actor collection

// Add materials with reference names to collection
PxCollection* materialCollection = physics->createCollection();

for(PxU32 i=0;i<mNbMaterials;i++)
    materialCollection->add(*materials[i], PxSerialObjectRef(i+1));

// Create actor collection, complete and serialize
PxCollection* actorCollection = physics->createCollection();

for(PxU32 i=0;i<mNbActors;i++)
    actorCollection->add(*actors[i]);

PxSerialization::complete(*actorCollection, materialCollection, *registry);

// Binary
    PxSerialization::serializeCollectionToBinary(actorStream, *actorCollection, *registry, materialCollection);
//~Binary

// RepX
    PxSerialization::serializeCollectionToXml(actorStream, *actorCollection, *registry, NULL, materialCollection);
//~RepX

actorCollection->release();
materialCollection->release();          // Note that materialCollection was not serialized
registry->release();

Deserialization:

PxPhysics* physics;                     // The physics SDK object
PxScene* scene;                         // The scene into which the objects will be inserted
PxSerializationRegistry* registry;      // Registry for Serializable types
PxCooking* cooking;                     // Cooking library needed for instantiating objects by repX
PxMaterial** materials;                 // Created procedurally by application

// recreate material collection with consistent reference names, no deserialization
PxCollection* materialCollection = physics->createCollection();

for(PxU32 i=0;i<mNbMaterials;i++)
    materialCollection->add(*materials[i], PxSerialObjectRef(i+1));

// Deserialize actors with reference material collection

// Binary
    void* actorMemory128;                // aligned memory containing serialized actors
    PxCollection* actorCollection = PxSerialization::createCollectionFromBinary(actorMemory128, *registry, materialCollection);
//~Binary

// RepX
    PxInputData& actorInputData = ...;   // Implemented by the application
    PxCollection* actorCollection = PxSerialization::createCollectionFromXml(actorInputData, *cooking, *registry, materialCollection);
//~RepX

materialCollection->release();
scene->addCollection(*actorCollection);
actorCollection->release();
registry->release();

Reference Counting of Deserialized Objects

This section assumes the background in Reference Counting.

Creating an object from PxPhysics with an initial reference count of 1 ensures that the object will never be destroyed unless the application gives up the reference by calling release(). The information whether the application gave up a reference to an object is not preserved on serialization. Objects that are created by deserialization are always created with a reference that the application needs to give up by calling release().

See Shapes and Geometries for a discussion of the method PxRigidActor::createShape, which automatically releases the initial reference to the shape, leaving only the actor's reference. Again, the information that this reference has been released is not preserved by serialization.

Example for shapes:

PxOutputStream& outStream;          // Output stream for the collection
PxSerializationRegistry* registry;  // Registry for Serializable types
PxRigidActor* actor;                // Any actor

// Creating shapes in different ways implies different rules for releasing
PxShape* shapeA = actor->createShape(...);           // Shape is automatically released when actor gets released
PxShape* shapeB = PxGetPhysics().createShape(...);   // Shape is either created as "shared" or "exclusive" and needs to be released by the applciation
actor->attachShape(*shapeB);                         // Attach shapeB to actor

// Create collection with actor and shapes and serialize
PxCollection* collection = PxGetPhysics().createCollection();
collection->add(*actor);
collection->add(*shapeA);
collection->add(*shapeB);
PxSerialization::serializeCollectionToBinary(outStream, *collection, *registry, NULL);
collection->release();

// Releasing actors and shapes
actor->release();    // Releases actor and shapeA (automatically)
shapeB->release();   // Releases shapeB (necessary since shapeB was created through PxPhysics)

// Deserialize collection
...
void* memory128;     // Aligned memory pointing to serialized data originally stored with outStream
collection = PxSerialization::createCollectionFromBinary(memory128, *registry, NULL);   // Restore collection

// Release actors and release ALL shapes (necessary since shape creation history is not preserved across serialization
for(PxU32 i = 0; i < collection->getNbObjects(); i++)
{
    switch ( collection->getObject(i).getConcreteType() )
    {
        case PxConcreteType::eRIGID_DYNAMIC:
        case PxConcreteType::eRIGID_STATIC:
            static_cast<PxActor&>(collection->getObject(i)).release();    // Doesn't release any shapes
            break;
        case PxConcreteType::eSHAPE:
            static_cast<PxShape&>(collection->getObject(i)).release();    // All shapes need to be released explicitly
            break;
    }
}

Reconnecting PhysX and Game-Objects

Here is an example of how to fix up references with gameplay objects by querying the reference names of a collection:

PxPhysics* physics;                      // The physics SDK object
PxCooking* cooking;                      // Cooking library needed for instantiating objects by repX
PxSerializationRegistry* registry;       // Registry for Serializable types

// Deserialize objects along with reference names

// Binary
    void* memory128;                     // Aligned memory containing serialized objects
    PxCollection* collection = PxSerialization::createCollectionFromBinary(memory128, *registry, NULL);
//~Binary

// RepX
    PxInputData& inputData = ...;         // Implemented by the application
    PxCollection* actorCollection = PxSerialization::createCollectionFromXml(actorInputData, *cooking, *registry, materialCollection);
//~RepX

// Receive a list of all deserialized reference names
#define MAX_REFS 100
PxSerialObjectRef refBuffer[MAX_REFS];
PxU32 numReferences = collection->getReferences(refBuffer, MAX_REFS);

// iterate over the list to patch up gameplay objects
for (PxU32 i = 0; i < numReferences; i++)
{
    PxActor* actor = collection->find(refBuffer[i])->is<PxActor>();
    if (actor)
    {
        // this assumes that findGamePlayObjectFromRef is able to locate
        // the corresponding game play object from a PxSerialObjectRef
        actor->userData = findGamePlayObjectFromRef(refBuffer[i]);
    }
}

Alternatively one could use PxCollection::getObjects(...) and PxCollection::getReference(PxBase& object) to achieve the same.

Serializing Everything

PhysX provides two utility functions for serializing the entirety of the PhysX runtime: PxSerialization::createCollection(PxPhysics& sdk) and PxSerialization::createCollection(PxScene& scene):

PxPhysics* physics;                 // The physics SDK object
PxScene* scene;                     // The physics scene
PxSerializationRegistry* registry;  // Registry for Serializable types
PxOutputStream& outStream;          // The user-defined stream doing the actual write to disk

// 1) Create a collection of all objects from the physics SDK.
PxCollection* everythingCollection = PxSerialization::createCollection(*physics);

// 2) Create a collection of all objects from the scene and add to everythingCollection.
PxCollection* collectionScene = PxSerialization::createCollection(*scene);
everythingCollection->add(collectionScene);
collectionScene->release();

// 3) Complete collection
PxSerialization::complete(*everythingCollection, *registry);

// 4) serialize collection and release it

// Binary
    PxSerialization::serializeCollectionToBinary(outStream, *everythingCollection, *registry);
//~Binary

// RepX
    PxSerialization::serializeCollectionToXml(outStream, *everythingCollection, *registry);
//~RepX

everythingCollection->release();
registry->release();

Deserialization is as previously:

PxScene* scene;                      // The physics scene
PxCooking* cooking;                  // Cooking library needed for instantiating objects by repX
PxSerializationRegistry* registry;   // Registry for Serializable types

// Binary
    void* memory128 = ...;           // a 128-byte aligned buffer previously loaded from disk by the user
    PxCollection* everythingCollection = PxSerialization::createCollectionFromBinary(memory128, *registry);
//~Binary

// RepX
    PxInputData& inputData = ...;     // Implemented by the application
    PxCollection* everythingCollection = PxSerialization::createCollectionFromXml(inputData, *cooking, *registry);
//~RepX

scene->addCollection(*everythingCollection);
everythingCollection->release();
registry->release();

Serializability

This section contains various definitions to describe serializability of a collection. Whether a collection can be successfully serialized and deserialized, optionally given an external references collection, can be queried by calling PxSerialization::isSerializable(...)

Requires

A serializable object A requires another serializable object B if A maintains a reference to B that needs to be re-established for successfully deserializing A. This implies that B needs to be deserialized before A.

Here is the table of the relationship requires of all PhysX objects:

joints require their actors and constraint
rigid actors require their shapes
shapes require their materials and mesh, if any
articulations require their links and joints
aggregates require their actors
cloth actors require their cloth fabric

At the API level "requires" is expressed with the interface PxSerializer::requires(PxBase& b, PxCollection& c).

Note

The PxSerializer can be created through the PxSerializationRegistry given a PxBase object and provides serialization functionaly. See Serialization of Custom Classes.

Root

Roots are serializable objects that can be instantiated without being owned by other objects. An articulation link for example can only be instantiated as part of its articulation.

Most object types are roots. The following types are not roots:

exclusive (non-shared) shapes
articulation links
constraints
cloth fabrics

Complete

A set of serializable objects C is complete relative to a set D if every object required by C is in C or D This means that we can deserialize C so long as we have D.

Serializable

Here is the complete set of requirements on a collection C with dependencies to D such that C can be serialized:

  • Every object in C is either a root or required by another object in C. (no orphans)
  • C is complete relative to D. (no dangling references)
  • There is no object in D that requires an object in C (no circular dependencies)
  • There is no reference r such that D.find(r) and C.find(r) are valid and not equal (no clashes)

Binary Serialization Specifics

Binary Serialization allows the creation of memory blocks from which PhysX can later construct objects. The PhysX runtime constructs the objects in place, making this an efficient mechanism for loading objects. You may instance collections of objects simply by making multiple copies of a memory block and deserializing them.

The data is specific to a platform and SDK version. When exported from the runtime it is always targeted at the platform on which it was created, although PhysX can retarget it at another platform in a post-processing step. This allows the conversion of binary assets from authoring platforms (Windows, MacOs and Linux) to other platforms. See Retargeting to other Platforms.

Memory Management

Management of memory blocks containing deserialized objects is left to users. It is the user's responsibility to:

  • allocate the memory block. Note that it must be properly aligned, to a PX_SERIAL_FILE_ALIGN (128) bytes boundary.
  • fill the block with serialized data, typically by loading it from disk.
  • deallocate the memory block when the objects within have been released by PhysX.

Although the user owns the memory block, the PhysX runtime owns any deserialized objects it contains. Concretely, calling release() on an object that was created by deserialization will cause its destructor to run, but will not deallocate its memory. If you deallocate the block before the destructors have run for all the objects it contains, the PhysX runtime will likely crash. For more information about how deserialized objects need to be released see Reference Counting of Deserialized Objects.

Object Names

Some objects, such as shapes and actors, can be given names using the PxShape::setName() and PxActor::setName() functions. The SDK does not own those names, i.e. the strings remain in user memory. When serializing objects, you may choose whether to preserve those names in the serialized data, or to discard them to create smaller data. If you choose to preserve the names, they will be serialized along with the objects themselves when calling PxSerialization::serializeCollectionToBinary(). On deserialization, the names will live within the user-provided memory block.

Use the 'exportNames' parameter of the PxSerialization::serializeCollectionToBinary() function to control this behavior.

Retargeting to other Platforms

Binary serialized data is platform-specific, and when serialized it always targets the platform on which it was created. The binary converter in the extensions library retargets data from one platform to another. So to deploy data for PS3, XBox etc, typically you will serialize on PC, then use the converter to retarget for each platform.

The converter requires meta-data for the source and target platforms, which contains information about the binary layout of objects for that platform. To obtain metadata, use the function provided in the extensions library for each platform:

void PxSerialization::dumpBinaryMetaData(PxOutputStream& stream, const PxPhysics& physics, PxSerializationRegistry& sr);

On each target platform, run it once and keep generated data around. Alternatively a set of pre-built binary meta data is included with the PhysX SDK at [path to installed PhysX SDK]/Tools/BinaryMetaData.

Assuming you have initialized the extensions library, convert data as follows:

PxSerializationRegistry* registry;      // Registry for Serializable types
PxErrorCallback* myErrorCallback;       // an error callback implemented by the application
PxInputStream& srcMetadata;             // metadata for the 'from' platform (e.g. PxDefaultFileInputData)
PxInputStream& dstMetadata;             // metadata for the 'to' platform

PxInputStream& srcAsset;                // stream containing source asset
PxU32 srcAssetSize;                     // size of the source asset
PxOutputStream& dstAsset;               // output stream for retargeted asset

PxBinaryConverter* converter = PxSerialization::createBinaryConverter(*registry, myErrorCallback);
converter->setMetaData(srcMetadata, dstMetadata);
converter->convert(srcAsset, srcAssetSize, dstAsset);

API-level Serialization (RepX) Specifics

RepX stands for Representation X and is the ASCII-XML serialization format for PhysX 3. This format is intended to be a user-level format meaning the data is in a format that matches the public API. It is also intended to be backwards compatible, thus assets saved in RepX for version 3.0 of PhysX will load in version 3.1, 3.2, etc. We do not intend for RepX to be used in performance critical or memory constrained situations.

Object Names

As opposed to binary serialization, the object names that can be specified with the PxShape::setName() and PxActor::setName() functions, are always included in the serialized format. On deserialization with PxSerialization::createCollectionFromXml(...) the names can be recovered by setting the PxStringTable parameter. Otherwise the application will have to re-establish the names again using e.g. PxActor::setName().

If PxStringTable parameter is set, the names will live within the memory which is allocated by the string table. Please make sure not to release the string table unless it can be guaranteed that the names will not be accessed.

Upgrading RepX Data

Upgrading RepX data from an older PhysX version to a newer one is easy. It happens implicitly when you deserialize old RepX data with a newer PhysX SDK and re-serialize the resulting PxCollection.

Example for upgrading a RepX stream:

PxPhysics* physics;                                       // The physics SDK object (e.g. PhxsX 3.3)
PxCooking* cooking;                                       // Cooking library needed for instantiating objects by repX
PxSerializationRegistry* registry;                        // Registry for Serializable types

PxDefaultFileInputData inputData(pathTo30RepXFile);       //load an older 3.x repx file
PxCollection* collection = PxSerialization::createCollectionFromXml(inputData, *cooking, *registry);

PxDefaultFileOutputStream outStream(pathToNewRepXFile);
PxSerialization::serializeCollectionToXml(outStream, *collection, *registry);

Caching Cooked Geometry Data

In order to facilitate faster instantiation of xml data, one can optionally configure the xml serialization to store the cooked triangle and convex mesh data along with the plain data. The cooked data caching can be enabled by passing a PxCooking instance into PxSerialization::serializeCollectionToXml(...). The cached cooked data is ignored on loading when the cooked format is outdated.

Serialization of Custom Classes

The PhysX serialization system is extendable to custom application types. The following documentation doesn't cover all aspects of the extension mechanisms. It can therefore be useful to look into implementation examples:

  • CustomPulleyJoint class in the SnippetExtension (Snippets/SnippetExtension/src)
  • PhysXVehicle library (PhysXVehicle/src)

Binary Serialization of Custom Classes

One can serialize and deserialize a custom class by following these steps:

  1. Define a PxConcreteType and type info for the custom class. Make sure its type value is unique.
  2. The custom class needs to inherit from PxBase and implement it's interface. Additionally it needs to call setConcreteType in the class constructor.
  3. Instance PxSerializerDefaultAdapter<T> and implement specialized methods where necessary.
  4. If retargeting to other platforms is needed, implement getBinaryMetaData().
  5. Register the adapter and meta data, see PX_NEW_SERIALIZER_ADAPTER , PxSerializationRegistry::registerSerializable and PxSerializationRegistry::registerBinaryMetaData.

For pointer members one needs to do the following (Note that reference members are currently not supported):

  1. If a pointer points to another PxBase object, add the object in the method requires().
  2. Export/import non PxBase data by implementing exportExtraData()/importExtraData().
  3. Resolve pointers in resolvePointers().

Example for a custom class:

 #include "PxSerialization.h"
 #include "PxSerialUtils.h"
 #include "PxMetaData.h"
 #include "PxSerializer.h"

 using namespace physx;

 const PxType customClassType = PxConcreteType::eFIRST_USER_EXTENSION;
 PX_DEFINE_TYPEINFO(CustomClass, customClassType);

 class CustomClass : public PxBase
 {
     friend class PxSerializerDefaultAdapter<CustomClass>;
 public:
     CustomClass()
     {
         setConcreteType(customClassType);
     }
     virtual ~CustomClass() {}
     CustomClass(PxRefResolver& v) : PxBase(v) {}

     //PxBase
     virtual const char* getConcreteTypeName() const { return "CustomClass"; }

     virtual bool isKindOf(const char* name) const
     {
         return !strcmp("CustomClass", name) || PxBase::isKindOf(name);
     }
     //~PxBase

     //PxSerializationRegistry::registerBinaryMetaData
     static void getBinaryMetaData(PxOutputStream& stream)
     {
         PX_DEF_BIN_METADATA_VCLASS(CustomClass)
         PX_DEF_BIN_METADATA_BASE_CLASS(CustomClass, PxBase)

         PX_DEF_BIN_METADATA_ITEM(CustomClass, PxRigidDynamic, mActor, PxMetaDataFlag::ePTR)
         PX_DEF_BIN_METADATA_ITEM(CustomClass, char, mBuf, PxMetaDataFlag::ePTR)
         PX_DEF_BIN_METADATA_ITEM(CustomClass, PxU32, mSize, 0)

         PX_DEF_BIN_METADATA_EXTRA_ITEMS(CustomClass, char, mBuf, mSize, 0, 0)
     }
     //~PxSerializationRegistry::registerBinaryMetaData

    private:
     PxRigidDynamic* mActor;    //add in requires
     char* mBuf;                //extra data
     PxU32 mSize;               //size of mBuf
 };

 //PxSerializerDefaultAdapter
 template<>
 void PxSerializerDefaultAdapter<CustomClass>::requires(PxBase& obj, PxCollection& c)  const
 {
     CustomClass* custom = obj.is<CustomClass>();
     PX_ASSERT(custom);
     c.addRequired(*custom->mActor);
 }

 template<>
 void PxSerializerDefaultAdapter<CustomClass>::exportExtraData(PxBase& obj, PxSerialStream& stream, void*) const
 {
     CustomClass* custom = obj.is<CustomClass>();
     PX_ASSERT(custom);
     stream.align(PX_SERIAL_DEFAULT_ALIGN);
     stream.storeBuffer(custom->mBuf, custom->mSize);
 }

 template<>
 char* PxSerializerDefaultAdapter<CustomClass>::importExtraData(PxBase& obj, char* address, PxU32& totalPadding) const
 {
     CustomClass* custom = obj.is<CustomClass>();
     PX_ASSERT(custom);
     address = PxAlignStream(address, totalPadding, PX_SERIAL_DEFAULT_ALIGN);
     custom->mBuf = address;
     address += custom->mSize;
     return address;
 }

template<>
bool PxSerializerDefaultAdapter<CustomClass>::resolvePointers(PxBase& obj, PxRefResolver& resolver, void*) const
{
     CustomClass* custom = obj.is<CustomClass>();
     PX_ASSERT(custom);
     bool isResolved = false;
     custom->mActor = static_cast<PxRigidDynamic*>(resolver.resolveAddress(custom->mActor, isResolved));
     return isResolved;
 }
 //~PxSerializerDefaultAdapter

 void registerCustomClassBinarySerializer(PxSerializationRegistry& registry)
 {
     registry.registerSerializable(customClassType, PX_NEW_SERIALIZER_ADAPTER(CustomClass));
     registry.registerBinaryMetaData(CustomClass::getBinaryMetaData);
 }

RepX Serialization of Custom Classes

One can serialize and deserialize a custom class by following these steps:

  1. Do the first three steps from Binary Serialization of Custom Classes. Methods in PxSerializer/PxSerializerDefaultAdapter<T> required exclusively for binary serialization may be left empty.
  2. Create a custom RepX serializer that implements the PxRepXSerializer interface. PxRepXSerializer is used to create an object from the xml file and write an object to the xml file. You can also use the class RepXSerializerImpl to inherit default implementations for some methods.
  3. Register both, the general serializer adapter and the repx serializer.
  4. RepX supports automatic reading and writing of class properties. To achieve this, you have to use clang to generate corresponding meta data: PhysX API Meta Data System.

Example for a custom class:

#include "SnRepXSerializerImpl.h"

const PxType customClassType = PxConcreteType::eFIRST_USER_EXTENSION;
PX_DEFINE_TYPEINFO(CustomClass, customClassType);

struct CustomClassRepXSerializer : public RepXSerializerImpl<CustomClass>
{
    CustomClassRepXSerializer(PxAllocatorCallback& inCallback) : RepXSerializerImpl<CustomClass>(inCallback) {}

    virtual PxRepXObject fileToObject(XmlReader& inReader, XmlMemoryAllocator& inAllocator,
        PxRepXInstantiationArgs& inArgs, PxCollection* inCollection)
    {
        //factory for CustomClass instance provided by application
        CustomClass* object = createCustomClass();

        //when using the PhysX API metadata system readAllProperties(...) can be used to read all properties automatically
        readAllProperties(inArgs, inReader, object, inAllocator, *inCollection);

        return createRepXObject(object);
    }

    virtual void objectToFileImpl(const CustomClass* obj, PxCollection* inCollection, XmlWriter& inWriter,
        MemoryBuffer& inTempBuffer, PxRepXInstantiationArgs&)
    {
        //when using the PhysX API metadata system writeAllProperties(...) can be used to save all properties automatically
        writeAllProperties(obj, inWriter, inTempBuffer, *inCollection);
    }

    //this can return NULL if fileToObject(...) is overwritten with a custom implementation.
    virtual CustomClass* allocateObject(PxRepXInstantiationArgs&) { return NULL; }
};

void registerCustomClassRepXSerializer(PxSerializationRegistry& registry)
{
    registry.registerSerializable(customClassType, PX_NEW_SERIALIZER_ADAPTER(CustomClass));
    registry.registerRepXSerializer(customClassType, PX_NEW_REPX_SERIALIZER<CustomClassRepXSerializer>));
}

Note

Implementing a PxRepXSerializer is currently not practical without including the internal PhysXExtension header "SnRepXSerializerImpl.h".

PhysX API Meta Data System

It produces a set of objects that are analogues of the interfaces and some descriptors in the PhysX system, all based on the public interface. The generator heuristically finds functions that start with get/set and, through a series of cascading rules, combines those into several types of properties.

Currently the generator supports the following property types

  • Basic property
    • {ptype} get{pname}() const;
    • void set{pname}( const ptype& prop ); //plus variations
    • read-only, write-only variants of above.
  • Range property
    • void get{pname}( {ptype}& lowEnd, {ptype}& highEnd );
    • void set{pname}( {ptype} lowEnd, {ptype} highEnd );
  • Indexed property
    • {ptype} get{pname}( enumType idx );
    • void set{pname}( enumType idx, const {ptype}& prop );
  • Dual indexed property (like above, but with two enumeration indexes).
  • Collection
    • PxU32 getNb() const;
    • PxU32 get( {ptype}* buffer, PxU32 count );
    • void set({ptype}* buffer, PxU32 count);

To use it, you have to create the following files:

  • CustomTypeExtensionAPI.h

    • Add all the types you want to export to gUserPhysXTypes in this file.
    • Add the unnecessary types to gAvoidedPhysXTypes. It will not generate meta data information for these types.
    • Be sure to append the included files for these types.
  • runClangForCustomType.bat

    • Set definition folder for these autogenerated files and set the source file in here.

    • Specify the filename of autogenerated files. Then it will generate the following files:

      include/CustomTypeAutoGeneratedMetaDataObjectNames.h
      include/CustomTypeAutoGeneratedMetaDataObjects.h
      src/CustomTypeAutoGeneratedMetaDataObjects.cpp
      
  • CustomTypeMetaDataObjects.h

    • One has to define CustomTypePropertyInfoName and include CustomTypeAutoGeneratedMetaDataObjects.h in this file. Then it will export the properties of the custom class if one includes this file for implementing the custom repx serializer.
  • CustomTypeMetaDataObjects.cpp

    • This file is optional. It is only required when you need custom properties.

Take PxVehicle as an example, here is the structure of these files, take Source/PhysXVehicle as root:

src/PhysXMetaData/include/PxVehicleMetaDataObjects.h
src/PhysXMetaData/src/PxVehicleMetaDataObjects.cpp
../../Tools/PhysXMetaDataGenerator/PxVehicleRepXExtensionAPI.h
../../Tools/PhysXMetaDataGenerator/runClangForPxVehicle.bat

After you run ../../Tools/PhysXMetaDataGenerator/runClangForPxVehicle.bat, it will autogenerate such files:
src/PhysXMetaData/include/PxVehicleAutoGeneratedMetaDataObjectNames.h
src/PhysXMetaData/include/PxVehicleAutoGeneratedMetaDataObjects.h
src/PhysXMetaData/src/PxVehicleAutoGeneratedMetaDataObjects.cpp
  1. PxVehicleRepXExtensionAPI.h: We use DisabledPropertyEntry for these properties which we don't want to export, CustomProperty for these properties which we want to customize and gUserPhysXTypes for general properties which we want to export.
  2. runClangForPxVehicle.bat: We set target directory as src/PhysXMetaData, and set target name as PxVehicle.
  3. PxVehicleMetaDataObjects.h: It defines the custom properties and includes PxVehicleAutoGeneratedMetaDataObjects.h
  4. PxVehicleMetaDataObjects.cpp: It implements the custom properties.

Note

It will write the properties defined in PxVehicleAutoGeneratedMetaDataObjects.h to repx file automatically if you include PxVehicleMetaDataObjects.h for your own repx serializer.

Take CustomPulleyJoint as an example, here is the structure of these files, take Snippets/SnippetExtension as root:

src/PhysXMetaData/include/PxSnippetExtensionMetaDataObjects.h
../../Tools/PhysXMetaDataGenerator/PxSnippetExtensionAPI.h
../../Tools/PhysXMetaDataGenerator/runClangForPxSnippetExtension.bat

After you run ../../Tools/PhysXMetaDataGenerator/runClangForPxSnippetExtension.bat, it will autogenerate such files:
src/PhysXMetaData/include/PxSnippetExtensionAutoGeneratedMetaDataObjectNames.h
src/PhysXMetaData/include/PxSnippetExtensionAutoGeneratedMetaDataObjects.h
src/PhysXMetaData/src/PxSnippetExtensionAutoGeneratedMetaDataObjects.cpp
  1. PxSnippetExtensionAPI.h: We use gUserPhysXTypes for general properties which we want to export, use gRequiredPhysXTypes for the propreties inherit from PhysX types.
  2. runClangForPxSnippetExtension.bat: We set target directory as src/PhysXMetaData, and set target name as PxSnippetExtension.
  3. PxSnippetExtensionMetaDataObjects.h: It defines the custom properties and includes PxSnippetExtensionAutoGeneratedMetaDataObjects.h

Note

It will write the properties defined in PxSnippetExtensionAutoGeneratedMetaDataObjects.h to repx file automatically if you include PxSnippetExtensionMetaDataObjects.h for your own repx extension.