TREEspan
File System
Features and Advantages
Applications
Device Support
Articles
Adaptable and Versatile File System
TREEspan File System (TSFS) is designed to be very adaptable, able to handle various types of applications and workloads efficiently. This is achieved in multiple ways, the first being the support of multiple storage media including NOR, NAND, SD and eMMC. The choice of media is important to tailor the selected media to the application requirements. Secondly, TSFS can leverage the features and performance characteristics of each media, offering excellent performance on every media but also taking advantage of each media’s strength. Finally, the file system configuration can be modified to tweak the performance and resource usage to a certain platform and workload.
Database or Database-Like Applications
TSFS can support database application design in two different ways: either as the back end for a full-blown database or as a lightweight database in and of itself. In the first case, TSFS files are used to store user data and, most importantly, indexes and other internal database metadata. In the second case, files are used as key/value stores where the position within the file is used as the key. This scheme remains efficient even for a huge key space because TSFS stores sparse files in a compact way.
Whether your application is based on a large database or a simple key/value store, it will likely translate into small read/write accesses spread across a vast portion of the available data space. This is what we call a random access workload and it is notoriously hard on file system performances. But TSFS has been designed from the ground up with this particular problem in mind. In fact, TSFS is highly optimized for small access workloads on systems with limited random access memory (RAM). In particular, TSFS can perform small random write accesses while maintaining a very low write amplification. Write amplification is inevitable, but it must be kept to a minimum, not only for obtaining the best possible overall throughput, but also to improve flash lifetime and lower energy consumption.
Another key aspect of database and database-like applications is consistency. Consider a simple application where a collection of records (say sensor readouts) is regularly updated along with rolling statistics of those records (median, average, standard deviation, etc.). In this case, the data is said to be consistent if the statistics truly reflect the accumulated records and vice versa. Thanks to transactions, TSFS makes sure it stays that way despite untimely interruptions such as power failures, media removal or other application-level external events such as communication errors. Set a transaction point (what we call committing) and TSFS will come back to it.
Real-Time Applications
The expression “real-time” refers to a vast array of embedded applications, sometimes with radically different requirements. The common requirement to all real-time applications though is some kind of upper bound on execution time. TSFS provides such an upper bound on every file, directory and snapshot operation. The tightness of theses upper bounds are configurable such that you get to control the fundamental trade-off between average and worst-case performances.
One particular aspect of flash file systems (and/or flash translation layers (FTL)) that causes headaches to embedded application designers is garbage collection. Simply put, garbage collection is the mechanism responsible for relocating and compacting valid data as used flash blocks need to be erased. The typical problem with garbage collection with regard to real-time requirements resides in the large and sudden amount of read/write accesses that it can generate beyond the control of the application. Obviously, such untimely access peaks are not desirable for time-constrained applications.
TSFS solves the garbage collection problem in two different ways. First, TSFS lets the application control the garbage collection timing. Through a dedicated API, the application can trigger garbage collection when it makes most sense given its a priori knowledge of file system access timings. For instance, if sensor readouts are being performed at regular intervals, the application can take advantage of possible idle time between readouts to perform garbage collection. Second, TSFS limits garbage collection-induced access peaks by spreading garbage collection over multiple write accesses. This last feature is key for real-time applications as it places a hard limit on execution time, independent of any direct intervention from the application itself.
Note that the choice of storage technology can have a huge impact on worst-case access latency. Most notably, managed flash devices often exhibit large and unpredictable access peaks. Although TSFS does provide high average read/write throughput on managed storage technologies, it cannot possibly make up for the poor worst-case performances. For applications with strict timing requirements, it is advisable to either go for bare flash or, at least, industrial grade managed flash offering precise timing guarantees.
Data Logging Applications
Sometimes at the core of the application, often as a peripheral feature (such as debugging and tracing), data logging is needed almost everywhere. In its purest form, a single file is gradually filled with incoming data, up to a predetermined maximum size. Then, the file may be archived, renamed, deleted and/or simply overwritten as new data is coming in. No matter how you do it, though, TSFS will handle it effortlessly.
In terms of asymptotic performance, file deletion and renaming are O(1) with respect to file size and O(log(n)) with respect to the number of files. Sequential write accesses are also O(1) with respect to file size. As for file archiving, TSFS snapshotting capabilities can be very helpful, especially when it comes to archiving multiple files at once. In this case, snapshotted files can be read from while being simultaneously updated, all this, without the need for complex synchronization schemes.
Audio/video Streaming Applications
Video surveillance systems, home theatres, professional recording equipment, industrial video inspection: many modern embedded applications require some kind of audio/video streaming capabilities. These audio/video applications most often translate in large sequential read/write accesses at the file level. To make the most out of these large accesses, TSFS uses variable extent sizes. Simply put, larger write accesses generate less metadata, therefore increasing net read and write performances. No user configuration is needed. Simply write the data as it comes in, TSFS will automatically adapt.
Besides, TSFS is designed to fully take advantage of the sequential nature of audio/video streaming. In fact, when the overall workload is dominated by sequential accesses (as is also the case for data logging applications), the amount of garbage collected data is minimized which, in turn, increases the net write throughput. As a result, audio/video streaming can be performed at speeds approaching the raw capabilities of the underlying storage device.
Low-power Applications
One of the most challenging aspects of low-power applications (outside of energy consumption) is overcoming the often limited amount of RAM available. With minimum RAM requirements in excess of 32 KiB, most flash file systems and translation layers simply don’t fit. Manage flash devices, on the other hand, have dedicated RAM along with a built-in flash controller. While this does usually decrease RAM requirements on the host side, this does nothing to improve overall energy consumption.
With a minimum RAM requirement of 4 KiB, TSFS can be deployed on even the most modest platform. And because it natively supports bare NAND flash, TSFS makes truly low-power designs a reality.