TREEspan File System

Embedded Transactional Flash File System

TREEspan File System™ (TSFS) is a high performance embedded transactional file system with native NOR and NAND Flash support for RTOS and bare-metal applications. TSFS provides developers with robust, fail-safe, data storage on virtually any MCU or SoC.

Checking All the Boxes

feat_icon_nand
Parallel NAND
feat_icon_sd
SD / eMMC
feat_icon_nor
NOR Flash
feat_icon_serial_nand
Serial NAND

All-Round Media Support

TSFS is a flash file system for all flash devices. Support includes raw NOR, raw SLC NAND (serial (QSPI) and parallel), SD cards and eMMCs, all supported by a single unified codebase with low-level flash management tasks, like error correction and wear-leveling, built into the file system core.

feat_icon_performance
High Random Throughput
feat_icon_real_time
Real-Time
feat_icon_mount_time
Fast Mount
feat_icon_ram_usage
Low-RAM Usage

Stellar Performances

TSFS is the result of an uncompromising pursuit of performance. Minimal write jitter limits buffering requirements and keeps memory usage as low as possible. High random access throughput ensures stable performances accross a wide spectrum of workloads.

feat_icon_fail_safe
Fail-Safe
feat_icon_transactional
Transactional

More than fail-safe: transactional

TSFS is fail-safe, which means that the integrity of the file system itself is protected in case of an unexpected interruption like power failure. It is also transactional, which means that

feat_icon_documentation
Pristine Documentation
feat_icon_c_code
Full C99 Source
feat_icon_drivers
Drivers Included
feat_icon_licence
Simple Royalty-Free Licensing

Embedded Storage Made Simple

TSFS comes in full C99 source form with pristine documentation, examples and all the needed device drivers.

Media Support

SD/eMMC

SD cards and eMMCs are an obvious choice for storing large amounts of data. They also deliver very high sequential access throughputs (in excess of 10MB/s in both directions) which is great for streaming applications. Performances, however, tend to drop under more complex workloads (as discussed here) and this is where the file system can really make a difference.
 
TSFS leverages the fact that SD card and eMMC are really at their best when accessed in large and contiguous sequences. More specifically,

  • TSFS produces strictly sequential write accesses at the device level, independant of the write pattern at the file level;
  • TSFS stores on-disk metadata in compact structures favoring large sequential read acceses and supporting efficient lookup operations and a high read throughput;
  • TSFS supports true “blind writes” (where no lookup is needed prior to writing) favoring uninterrupted sequences of write accesses and a higher write throughput.
 

NOR Flash

Embedded systems with modest data storage requirements often have both code and data on a single shared NOR flash device. This is typically the case in deeply embedded applications where RAM is rare and precious. In such a memory-constrained environment, a file system might not even be needed. Otherwise, it must certainly have a low RAM footprint.

With native support for raw flash memory, TSFS does not require block device emulation. No extra adaptation software is needed, avoiding buffer and data structure duplication, and keeping RAM consumption at a minimum. With 4KiB of RAM, TSFS provides a complete file system abstraction while carrying crucial flash mamangement tasks such as garbage collection and wear-leveling.

TSFS also takes advantage of NOR flash support for arbitrarily small accesses with very low overhead. This makes a real difference for read-centric applications seeking to benefit from NOR flash jitter-free access times and near-bus speed read throughputs.

 

SLC NAND Flash

SLC NAND is a great fit for so many embedded applications. It has it all: low cost per bit, low energy consumption, and great overall performances. In its serial, semi-managed form, with built-in ECC, you get much of SLC NAND performance potential with very minimal integration effort. At the other end of the spectrum, the bare form with parallel interface provides the highest performance through pipelining of flash array access and bus transfer, and support for multi-plane operations.

One unsung merit of SLC NAND is its potential for extremely low write latency. TSFS taps into this potential
to provide strict timing guarantees and enable real-time applications on memory-constrained platforms.

Performance

High Performance Across the Board

The table below gives a summary of TSFS file read and write performances for various platforms and storage device. TODO: Explain benchmark and relevance.
perf_table

High Performance Across the Board

TSFS garbage collection optimizes write amplification, meaning that a minimal amount of data is collected at any given moment in order to free and recycle partially used data blocks (explained in details here).

A comparison of experimental and optimal (calculated) random write throughput values confirms how close TSFS is from the upper bound (right-hand side graph).

Note that optimal write throughput also means minimal write amplification, cell wear and energy consumption.

Optimal throughput (in green) calculated from experimental throughput values of the Mircron MT29F2G01 QSPI SLC NAND and based on the assumption of perfect garbage collection and the absence of any other form of overhead. Measurements are performed on the STM32F746ZG@200MHz.

perf_graph

Fast Mount

TSFS achieves transactionality using copy-on-write strategies (as opposed to relying on some form of journaling approach). As a result, mount cycling is read-only and extremently fast (left-hand side table).

Strict Real-Time Behaviour (thus Low-RAM)

Embeded systems commonly require that data be persistently recorded at a constant interval and without loss. In that case, minimizing write latency is key to achieving high average throughputs while keeping RAM usage under control.

Quick maths: assuming a 1MB/s constant input rate and an SD card with a quite typical 200ms maximum access time, we need at least 1MB/s * 0.2s = 200KB of RAM to avoid data loss.

By contrast, a real-time flash file system like TSFS achieves high throughputs with little to no buffering on raw NAND (see our article on the topic). On the Micron MT29F1G01 QSPI SLC NAND (right-hand side graph), TSFS can accomodate a 2MB/s input rate without loss with less than 6KiB of buffering RAM.

real_time_graphs

Fail-Safety and Transactions

How it Works

TSFS is a transactional file system. In a nutshell, it means that all modifications to files and directories are only effective after a commit operation successfully completes. The user initiates a commit operation by calling tsfs_commit(). Because the commit operation is atomic with respect to file system failures, one of two scenarios are possible:

 

  • the failure occurs after the commit such that the transaction completes without interruption and the file system state includes the latest updates;
  • the failure occurs before the commit and the file system state upon restarting is found in the state that it was in at the time of the latest (successful) commit.
tsfs-file-system-transaction-fail-safe

A Simple Example

A new frame of sensor values is recorded into a dedicated file at regular interval. Upon reaching a certain size threshold the file is archived and a separate archive index file is updated.

A robust solution using TSFS is illustraded below. In case of a power failure, the state of the file system after restarting would be either that of Commit 0 or Commit 1.

It would be impossible for instance to end up with a partial frame, or with a full data file for which no archive was created. Or to find an archive file without the corresponding entry in the archive index file. All of these corner cases, the application does not have to care about.

tsfs-file-system-transaction-example

Flash Management for raw NOR / NAND

Garbage Collection

The need for garbage collection stems from fundamental fash memory characteristics, and restrictions as to how it can be accessed. In a nutshell, flash memory is divided into large erase blocks, each of which is in turn divided into pages. One requirement is that a whole erase block must be erased before a page can be written.

Before it can be erased, a used block must first be freed, which means that any remaining bits of valid data must be moved (copied) to some alternate location (figure on the right-hand side).

This process, know as garbage collection, is built into TSFS core (as opposed to a separate FTL), which significantly reduces related RAM and I/O overhead. As pointed out in this article TSFS garbage collection is optimal.

tsfs-garbage-collection

Static and Dynamic Weal-Levelling

TSFS implements both dynamic and static wear-leveling. Dynamic wear-leveling is a straightforward extension of garbage collection as it operates only on free blocks. Static wear-leveling does a little bit more, actively freeing used data blocks otherwise. In both cases, the goal is the same: ensuring even distribution of cell wear such as to avoid premature block failures.

Bad Block Management (NAND Flash)

Although wear-leveling goes a long way towards avoiding premature block failures, flash blocks inevitably wear out at some point. When a block fails TSFS moves its content (from previous successfull page programing) to a spare block reserved for this purpose and remaps further accesses accordingly.

Error Correction (NAND Flash)

NAND flash is characterized by non-negliable bit error rates which must be compensated for by appropriate error correction algorithms. TSFS provides support for software ECC implementations, on-chip ECC such as those commonly found on serial NAND devices as well as host-side hardware ECC controllers such as NXP BCH controller and Texas Intrument Error Location Module (EML).

Everything Needed to Get You Going

Static and Dynamic Weal-Levelling

TSFS is distributed in C99 compliant source form, including drivers and support modules. All provided sources follow or exceed industry standard coding practices, including

  • Systematic error handling. Error codes are either handled in-place or returned intact to the caller. Error handling always includes appropriate clean-up code (mostly for releasing resources) such that both the file system and the surrounding runtime environment can resume in the event of a timeout, IO error or other non-fatal errors.
  • Defensive coding style with abundant checks and assertions including validation of function arguments, validation of external data (e.g. on-disk data structures), assertion of various expected internal states or conditions, and various bounds checking.
  • MISRA compliance. With supporting documentation including a detailed compliance matrix and corresponding rationale for possible deviations.

Example Application and Validation Test Suite

TSFS comes with a fully functional and thoroughly documented example application on the platform of your choice, making for an effortless startup experience. It also comes with a full validation suite, including file system and device driver benchmarks and functional tests, to make sure that you do get the most out of TSFS.

Documentation