TREEspan File System
Fail-Safe Embedded File System for Flash Memory
TREEspan File System™ (TSFS) is a high performance transactional embedded file system designed for RTOS and bare-metal applications. TSFS provides developers with robust, fail-safe, data storage on virtually any MCU or SoC and practically any type of embedded storage available.
TSFS natively supports both managed Flash devices such as SD Card and eMMC, and unmanaged Flash devices like NOR and NAND Flash, including parallel and serial (QSPI, OSPI etc.) variants. Reference drivers for flash devices as well as storage controllers are provided with the TSFS source distribution.
Details
| RAM Usage | 8kiB |
| ROM Usage | 32-64kiB |
| Mount Time | 20ms (NOR), 200ms (NAND), ~1000ms (SD/eMMC) |
| Performance | See Performance section. |
| RTOS Support | Bare-metal, FreeRTOS, PX5, Zephyr, ThreadX, uC/OS |
| Wear Levelling | Static and Dynamic wear levelling |
| NOR Support | Parallel and Serial QSPI/xSPI NOR |
| NAND Support | Parallel and Serial QSPI/xSPI SLC & MLC NAND |
| SD/eMMC Support | 4 & 8 lanes with USH/HS mode support |
Documentation
user guide
data sheet
Features and Advantages
- Transactional Fail Safe File System
- Native NOR, NAND, SD/eMMC Flash Support
- High random read/write performance
- Fast Mount Time
- Bounded Real Time Operations Latency
- C99 Source Code
- Pristine documentation
What’s Included
- Source code
- Reference drivers
- Integration test code and benchmarks
- Documentation
Base License
Included with every licensed distribution of TSFS:
- TSFS File System source code
- Royalty free license
- Reference device drivers for SD Card, eMMC, NOR and NAND Flash.
- Integration tests and benchmarks
- 1 year of support and update
$16k
Multi-Product License
In addition to the base license:
- Customized license for multiple products or MCUs
- Tailored support
- Reference drivers for covered media and controllers
Contact Us
TSFS for Certification
In addition to the base license:
- Safety-related requirements and documentation
- Unit and Functional tests
- Stress tests including power failures and I/O error recovery tests
- Timing tests and advanced benchmarks
Contact Us
Media Support
SD / MMC
SD cards and eMMCs are an obvious choice for storing large amounts of data. They also deliver very high sequential access throughputs (in excess of 10MB/s in both directions) which is great for streaming applications. Performances, however, tend to drop under more complex workloads (as discussed in our SD Card benchmark article) and this is where an optimized embedded file system can really make a difference.
TSFS leverages the fact that SD card and eMMC are really at their best when accessed in large and contiguous sequences. More specifically,
- TSFS produces sequential write accesses at the device level, independent of the write pattern at the file level;
- TSFS stores on-disk metadata in compact structures favouring large sequential read access and supporting efficient lookup operations and a high-read throughput;
- TSFS supports true “blind writes” (where no lookup is needed prior to writing) favouring uninterrupted sequences of write accesses and a higher write throughput.
NOR Flash
Embedded systems with modest data storage requirements often have both code and data on a single shared NOR flash device. This is typically the case in deeply embedded applications where RAM is rare and precious. In such a memory-constrained environment, a file system might not even be needed. Otherwise, it must certainly come with minimal RAM requirements.
With native support for raw flash memory, TSFS does not require block device emulation. No extra adaptation software is needed, avoiding buffer and data structure duplication, and keeping RAM consumption at a minimum. With 8KiB of RAM, TSFS provides a complete file system abstraction while carrying crucial flash management tasks such as garbage collection and wear-levelling.
TSFS also takes advantage of NOR flash support for arbitrarily small accesses with very low overhead. This makes a real difference for read-centric applications seeking to benefit from NOR flash jitter-free access times and near-bus speed read throughput.
NAND Flash
SLC NAND is a great fit for so many embedded applications. It has it all: low cost per bit, low energy consumption, and great overall performances. In its serial, semi-managed form, with built-in ECC, you get much of SLC NAND performance potential with very minimal integration effort. At the other end of the spectrum, the bare form with parallel interface provides the highest performance through pipelining of flash array access and bus transfers, and support for multi-plane operations.
One unsung merit of SLC NAND is its potential for extremely low write latency. TSFS taps into this potential
to provide strict timing guarantees and enable real-time applications on memory-constrained platforms (see Strict Real-Time Behaviour). On parallel NAND with dual-plane operations, for instance, TSFS can produce 5MB/s of sustained random write throughput with a total footprint (TSFS + application-level buffering) of 24KiB.
Performances
High Performances Across the Board
As much as we love to discuss file system performance, sometimes it’s best to let the numbers do the talking. Here is a sample of performance measurements on key combinations of platforms and storage devices. All accesses are 4KiB in size and the file system is half full in all cases.
Optimal Random Write Throughput
TSFS garbage collection optimizes write amplification, meaning that a minimal amount of data is collected at any given moment in order to free and recycle partially used data blocks (explained in our article on Flash File System Performance).
A comparison of experimental and optimal (calculated) random write throughput values confirms how close TSFS is from the upper bound (see graph). Minimal write amplification not only means higher average throughput but also lower cell wear (and thus extended lifetime) and lower energy consumption.
Optimal throughput (in green) calculated from experimental values of the Mircron MT29F2G01 QSPI NAND. Based on the assumption of perfect garbage collection and the absence of any other overhead. Measurements performed on the STM32F746ZG@200MHz.
Low RAM Usage
TSFS provides a high performance file system abstraction with minimal RAM footprint. All numbers given here include internal buffers. Some additional buffering at the application level might be needed, although TSFS offers strict timing guarantees minimizing the amount of extra buffering required (see Strict Real-Time Behaviour below).
Strict Real-Time Behaviour
Embedded systems commonly require that data be persistently recorded at a constant time interval and without loss. Minimizing write latency is key to meeting such a requirement while preserving a high average throughput and keeping RAM usage under control (see our article on Embedded File System RAM Usage).
Assuming a 1MB/s constant input rate and an SD card with a typical 200ms maximum access time, we need at least 1MB/s * 0.2s = 200KB of RAM to avoid data loss.
By contrast, a real-time flash file system like TSFS achieves high throughput with little to no buffering on raw NAND. On the Micron MT29F1G01 QSPI SLC NAND, TSFS can accommodate a 2MB/s input rate without loss with only 6KiB of buffering RAM (see graph).
Fast Mount Time
TSFS achieves transactionality using copy-on-write strategies (as opposed to relying on some form of journalling approach). As a result, mount cycling is read-only and extremely fast.
Fail-Safety and Transactionality
Transactional is More Than Fail-Safe
Fail-safe means that TSFS will gracefully recover from any form of untimely interruption (power failure, device removal, or else). This is different from transactional, which adds on top of fail-safety, guaranteeing that all data stored in the file system will be returned to its latest consistent state, as specified by the application (see next section). Having transactional logic baked into the file system takes the difficult and critical task of implementing and testing one-off recovery logic off of the application designer’s plate.
How it Works
As far as coding goes, TSFS is very much like any other embedded file system, the most notable addition being the tsfs_commit() interface. Any modification to the file system (file write, file deletion, directory creation, etc.) is not effective as long as tsfs_commit() does not successfully complete. Modifications performed after committing are discarded upon recovery. Crucially, committing is atomic, meaning that an interruption can either occur before or after, but not “during” a commit.
This all-or-nothing behaviour means that interruption timing with regard to the ongoing transaction has no impact on the final outcome (see figure).
All that’s left to the application designer is to judiciously define what goes into a transaction such that it makes sense for the application and cuts possible recovery paths down to just a few, easy-to-handle cases.
A Simple Example
Say a new frame of samples is appended to a file at a regular interval. Upon reaching a certain size threshold, the file is archived and a separate archive index file is updated accordingly. A robust solution using TSFS is shown below. If a power failure were to occur anywhere between Commit 0 and Commit 1, all the files would be returned to whatever they were at the time of Commit 0.
Notice how the content of the transaction is purposely defined such that, upon recovery, it would be impossible to find a partial frame, for instance, or a full data file for which no archive was created, or an archive file without the corresponding entry in the archive index file. Ruling out all these partial update scenarios, there is not much left to do for the application in terms of recovery, as the application will always restart from a coherent state, independent of the failure timing.
Flash Management for Raw NOR / NAND
Garbage Collection
The need for garbage collection stems from fundamental flash memory characteristics and restrictions. In a nutshell, flash memory is divided into large erase blocks, each of which in turn divided into pages. One requirement is that a whole erase block must be erased before a page can be written. An embedded file system must take into consideration these fundamental characteristics of flash memory to offer both performance and reliability.
Before it can be erased, a used block must first be freed, which means that any remaining bits of valid data must be moved (copied) to some alternate location (see figure).
This process, known as garbage collection, is built into TSFS core (as opposed to a separate FTL), which significantly reduces related RAM and IO overhead. In fact, TSFS garbage collection is optimal (see our article on flash file system performance).
Static and Dynamic Wear-Levelling
TSFS implements both dynamic and static wear-levelling. Dynamic wear-levelling is a straightforward extension of garbage collection as it operates only on free blocks. Static wear-levelling does a little bit more, actively freeing long-standing data blocks that would likely never be collected otherwise. In both cases, the goal is the same: ensuring even distribution of cell wear such as to avoid premature block failures.
Bad Block Management for NAND Flash
Wear-levelling goes a long way towards avoiding premature block failures. Regardless, flash blocks will inevitably wear out at some point. When a block fails (that is, when a program or erase operation fails) TSFS moves its content (from previous successful page programing) to a spare block reserved for this purpose and remaps further accesses accordingly.
Error Correction for NAND Flash
NAND flash is characterized by some bit error rate which must be compensated for by appropriate error correction algorithms. TSFS provides support for software ECC implementations, on-chip ECC such as those commonly found on serial NAND devices as well as host-side hardware ECC controllers such as NXP BCH controller and Texas Instrument Error Location Module (ELM).
Further Reading
What Is a Flash File System?
NOR vs NAND: So You Think You Know the Music?
