Data Science

Analyzing Cassandra Data using GPUs, Part 2

Oct 14, 2021
By Alex Cai
Discuss (0)

AI-Generated Summary
  • Sstable-to-arrow parses Cassandra SSTable files and converts them to Apache Arrow format so the data can be loaded into cuDF for GPU-accelerated analytics.
  • The tool is written in C++17 and uses the Kaitai Struct library to declaratively describe SSTable layouts, which the Kaitai compiler turns into C++ parsing classes.
  • Current limitations include support only for Cassandra OSS 3.11 SSTables, a maximum of 64 columns per table, inability to process SSTables larger than host memory, and lack of support for counter, frozen, and user-defined CQL types.
  • Decimal values are converted to 8-byte floating point numbers, sets are treated as lists, and varints larger than 8 bytes cause a crash.
  • The roadmap aims to add a read_sstable function to the RAPIDS ecosystem, broaden CQL type support, enable large-dataset handling, and parallelize SSTable reading for better GPU utilization.
  • Sstable-to-arrow can also export SSTable data to Parquet files using the -p flag, though deduplication is not yet performed.

Next Steps

  • Run the Docker image to start the sstable-to-arrow server on port 9143.
  • Build the project from source by following the instructions in the GitHub repository.
  • Download sample IoT datasets from the Google Drive folder to experiment with the tool.
Powered by NVIDIA Nemotron. AI-generated content may summarize information incompletely. Verify important information. Learn more

Editor’s Note: Watch the Analysing Cassandra Data using GPUs workshop.

In the previous post, I talked about our journey to find the best way to load SSTable data onto the GPU for data analytics. We looked at various methods of converting Cassandra data into formats usable by RAPIDS and decided to create sstable-to-arrow, a custom implementation that parses sstables and writes them to arrow format. In this post, we’ll discuss sstable-to-arrow further, its capabilities, limitations, and how to get started with it for your analytics use cases.

Implementation details

Sstable-to-arrow is written in C++17. It uses the Kaitai Struct library to declaratively specify the layout of the SSTable files declaratively. The Kaitai Struct compiler then compiles these declarations into C++ classes, which can be included in the source code to parse the SSTables into data in memory. Then, it takes the data and converts each column in the table into an Arrow Vector. Sstable-to-arrow can then ship the Arrow data to any client, where the data can be turned into a cuDF and be available for GPU analysis.

Current limitations

  • SStable-to-arrow can only read one SSTable at a time. To handle multiple SSTables, the user must configure a cuDF for each SSTable and use the GPU to merge them based on last write wins semantics. 
  • sstable-to-arrowexposes internal cassandra timestamps and tombstone markers so that merging can be done at the cuDF layer.
  • Some data, including the names of the partition key and clustering columns, can’t actually be deduced from the SSTable files since they require the schema to be stored in the system tables.
  • Cassandra stores data in memtables and commitlogs before flushing to SSTables, so analytics performed using only sstable-to-arrow will potentially be stale / not real time.
  • Currently, the parser only supports files written by Cassandra OSS 3.11.
  • The system is set up to scan entire SSTables (not read specific partitions). More work will be needed if we ever do predicate pushdown.
  • The following cql types are not supported: counter, frozen, and user-defined types.
  • varints can only store up to 8 bytes. Attempting to read a table with larger varints will crash.
  • The parser can only read tables with up to 64 columns.
  • The parser loads each SSTable into memory, so it is currently unable to handle large SSTables that exceed the machine’s memory capacity.
  • Decimals are converted into an 8-byte floating point value because neither C++ nor Arrow has native support for arbitrary-precision integers or decimals like the Java BigInteger or BigDecimal classes. This means that operations on decimal columns will use floating point arithmetic, which may be inexact.
  • Sets are treated as lists because Arrow has no equivalent of a set.

Roadmap and future developments

The ultimate goal for this project is to have some form of a read_sstable function included in the RAPIDS ecosystem, similar to cudf.read_csv. Performance is also a continuous area of development, and I’m currently looking into ways in which reading the SSTables can be further parallelized to make the most use of the GPU. I’m also working on solving or improving the limitations addressed preceding, especially broadening support for different CQL types and enabling the program to handle large datasets.

How to Use sstable-to-arrow

You can run sstable-to-arrow using Docker.

This will listen for a connection on port 9143. It expects the client to send a message first, and then it will send data in the following format:

  • The number of Arrow Tables being transferred as an 8-byte big-endian unsigned integer
  • For each table:
    • Its size in bytes as an 8-byte big-endian unsigned integer.
    • The contents of the table in Arrow IPC Stream Format.

You can then fetch the data from your port using any client. To get started with the sample Python client, follow these steps if your system does not support CUDA:

If your system does support CUDA, we recommend creating a conda environment with the following commands. You will also want to pass the -x flag when starting the sstable-to-arrow server preceding to turn all non-cudf-supported types into hex strings.  

To experiment with other datasets, you will want raw SSTable files on your machine. You can download sample IOT data at this Google Drive folder. You can also generate IOT data using the generate-data script in the repository, or you can manually create a table using CQL and the Cassandra Docker image (see the Cassandra quickstart for more info). Make sure to use Docker volumes to share the SSTable files with the container:

You can also pass the -h flag to get information about other options. If you would like to build the project from the source, follow the steps in the GitHub repository.

SSTable to Parquet

Sstable-to-arrow is also able to save the SSTable data as a Parquet file, a common format for storing columnar data. Again, it does not yet support deduplication, so it will simply output the sstable and all metadata to the given Parquet file.

You can run this by passing the -p flag followed by the path where you would like to store the Parquet file:

Conclusion

Sstable-to-arrow is an early but promising approach for leveraging Cassandra data for GPU-based analytics. The project is available on GitHub and accessible through Docker Hub as an alpha release.

We will be holding a free online workshop, which will go deeper into this project with hands-on examples in mid August! You can sign up here if you’re interested.

If you’re interested in trying out sstable-to-arrow, look at the second blog post in this two-part series and feel free to reach out to seb@datastax.com with any feedback or questions.

Discuss (0)

Tags

Data Science | RAPIDS | featured

About the Authors

Avatar photo
About Alex Cai
Alex Cai is a 2021 intern at DataStax and a student in the Harvard Class of 2025. He’s enthusiastic about computers, software, and cognitive science, and in his spare time, he enjoys reading, researching linguistics, and playing with his cat.

Comments