A parallel file system offers several advantages over a single direct attached file system. By using fast, scalable, external disk systems with massively parallel access to data, researchers can perform analysis against much larger datasets than they can by batching large datasets through memory. When a parallel file system is used in conjunction with the in-memory database kdb+, the advantages are:
1. A significant decrease in operational latency per kdb+ query, especially when running queries that search through significant amounts of historical market information. Achieved by balancing content around multiple file system servers.
2. Parallelization of kdb+ query “threads” in a single shared namespace, allowing a user to treat any data workload independently from other data workloads.
3. Supports simultaneous read/write operations on a single namespace for the entre database and for any number of kdb+ clients, (i.e.: end of day data consolidations into a hdb instance)
4. Allows for sharing of data among different independent hdb/rdb instances. Many instances of kdb can view the same data, meaning that strategies for data sharing and private data segments may be consolidated onto the same space. Avoids the need for kdb+ admininstartors to physically copy data around the network or disks, as automatic space allocation balancing is built-in to the parallel FS.
5. kdb+ context can be “striped” around all FS servers, or can be allocated in a round-robin fashion against each server. Striping allows the opportunity for some files to attain maximal I/O rates for a single kdb+ “object”.
To learn more about the advantages of a Parallel File Systems download this guide
All information that you supply is protected by our privacy policy. By submitting your information you agree to our Terms of Use.
* All fields required.
