Skip to content

financefs is a powerful Go FUSE filesystem that turns Yahoo Finance into a local cache. Access all historical ticker data (daily, weekly, monthly) as clean CSV files by simply navigating a directory structure, drastically reducing server load and protecting your IP from throttling.

License

Notifications You must be signed in to change notification settings

pluto-org-co/financefs

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

26 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

πŸ“ˆ financefs: A FUSE Filesystem for Financial Data

financefs is a FUSE (Filesystem in Userspace) implementation that virtualizes historical stock and financial data from Yahoo Finance as a local directory structure.

The primary purpose of financefs is data caching. By automatically caching retrieved data as CSV files, it significantly reduces repeated requests to Yahoo Finance servers, helping to prevent potential IP blacklisting due to excessive load.


πŸ› οΈ Installation

Prerequisites

To build and install financefs from source, you must have the following installed:

  • Go: Version 1.25 or later.

Install from Source

Execute the following command in your terminal. This command uses the jsonv2 experiment flag, which is required for proper installation of the project's dependencies.

GOEXPERIMENT=jsonv2 go install github.com/pluto-org-co/financefs/cmd/financefs@latest

πŸš€ Usage

1. Mounting the Filesystem

After installation, execute the binary with the desired mount point path. Ensure the mount point directory exists.

# Example: Using a directory in your home folder
mkdir ~/finance_data
financefs ~/finance_data

The system will now be mounted at ~/finance_data.

2. Accessing Data

Files under the mounted directory represent cached financial tickers (e.g., AAPL, ^SPX).

πŸ“ Filesystem Structure

Path Description
mountpoint/README.md Documentation file.
mountpoint/TICKER/ A directory representing a cached financial ticker (e.g., MSFT).
mountpoint/TICKER/daily.csv All-time historical data with a daily resolution.
mountpoint/TICKER/weekly.csv All-time historical data with a weekly resolution.
mountpoint/TICKER/monthly.csv All-time historical data with a monthly resolution.

On-Demand Caching

If a ticker is not visible, it means it is not yet cached. Simply accessing its directory will trigger the data pull and cache creation:

# Listing the mountpoint might not show 'F' (Ford) yet
ls ~/finance_data

# Accessing the directory pulls the data and creates the files
cd ~/finance_data/F

# Now the files are available
ls ~/finance_data/F
# Output: daily.csv  monthly.csv  weekly.csv

Retrieving Data

Use standard file system commands to retrieve the CSV data:

# Get all-time daily data for AAPL
head ~/finance_data/AAPL/daily.csv

3. Unmounting the Filesystem

To safely unmount the filesystem, use the standard umount command:

umount /path/to/mountpoint
# Example:
umount ~/finance_data

πŸ’» Data Analysis Example (Python/Pandas)

Since the data is presented as standard CSV files, it integrates seamlessly with popular data analysis tools like Pandas.

Here is a small Python program that loads the daily data for a ticker and displays the first few rows:

import pandas as pd
import os

# Define the mount point and the ticker you want to analyze
MOUNT_POINT = "~/finance_data"
TICKER = "AAPL"

# Construct the file path using the os.path.expanduser to handle the tilde (~)
data_path = os.path.expanduser(f"{MOUNT_POINT}/{TICKER}/daily.csv")

try:
    # Attempt to read the CSV file into a Pandas DataFrame
    df = pd.read_csv(data_path, index_col='time', parse_dates=True)

    print(f"βœ… Successfully loaded daily data for {TICKER}:")
    print("-" * 40)
    print(df.head())

except FileNotFoundError:
    print(f"❌ Error: File not found at {data_path}. ")
    print(f"Make sure {TICKER} has been accessed on the mounted system.")
except Exception as e:
    print(f"An unexpected error occurred: {e}")

Outputs:

βœ… Successfully loaded daily data for AAPL:
----------------------------------------
                           # source exchange name symbol interval currency          timezone  open  high  low  close     volume
time
1980-12-12 09:30:00-05:00  0  yahoo      NasdaqGS   AAPL    daily      USD  America/New_York    12    12   12     12  469033600
1980-12-15 09:30:00-05:00  1  yahoo      NasdaqGS   AAPL    daily      USD  America/New_York    12    12   12     12  175884800
1980-12-16 09:30:00-05:00  2  yahoo      NasdaqGS   AAPL    daily      USD  America/New_York    11    11   11     11  105728000
1980-12-17 09:30:00-05:00  3  yahoo      NasdaqGS   AAPL    daily      USD  America/New_York    11    11   11     11   86441600
1980-12-18 09:30:00-05:00  4  yahoo      NasdaqGS   AAPL    daily      USD  America/New_York    11    11   11     11   73449600

πŸ“„ License

This project is released under the GNU AFFERO GENERAL PUBLIC LICENSE (AGPL-3.0). See the LICENSE file for more details.

About

financefs is a powerful Go FUSE filesystem that turns Yahoo Finance into a local cache. Access all historical ticker data (daily, weekly, monthly) as clean CSV files by simply navigating a directory structure, drastically reducing server load and protecting your IP from throttling.

Topics

Resources

License

Stars

Watchers

Forks