L2-16: IPFS Node (Setting Up an IPFS Node) #

1. Problem #

Most NFT projects' "metadata storage" claims to be decentralized, but in practice heavily depends on centralized pinning services like Infura's IPFS gateway and Pinata. Your NFT image's "permanent storage" is actually just "a company promises to pin data for you" -- if that company discontinues service, changes pricing, or experiences downtime, your NFT metadata is no longer reachable. While the data may still exist somewhere on the IPFS network (if someone else has pinned it), reachability is not guaranteed.

Setting up your own IPFS node gives you full control over NFT data storage: content is served by your node (not a third-party gateway), pin policies are decided by you (which content needs permanent retention), and you can provide a public IPFS gateway service for the community. This is the shift from "depending on others' infrastructure" to "contributing your own infrastructure."

This challenge (refer to scripts/ipfs-node.md) requires completing the full process of setting up an IPFS node from scratch -- including installation, initial configuration, content pin management, garbage collection policies, and integration with Solidity NFT contracts.

2. Why #

IPFS (InterPlanetary File System) is a peer-to-peer distributed file system that uses Content Addressing rather than Location Addressing. In HTTP, you access a file via a URL (like https://example.com/image.png) -- the URL tells you "where the file is." In IPFS, you access a file via a CID (Content Identifier, like QmXxXxX...) -- the CID tells you "what the file is," because the CID is the hash of the file's content.

Content addressing implies two key properties:

1. Immutability: The same file content always produces the same CID; changing one byte of the file produces an entirely new CID. This ensures the integrity of NFT metadata -- you can verify that the data you received is truly the original version.
2. Decentralization: Anyone can "serve" content with a given CID -- your node, a friend's node, a public gateway -- as long as a node holding that content can be found through the DHT (Distributed Hash Table).

Running your own IPFS node transforms you from a "consumer" into a "provider": you not only consume content from the IPFS network, but your node also contributes storage and bandwidth to the network. When your node holds content for a given CID, it responds to requests from other nodes in the network -- you become part of the IPFS infrastructure.

In the context of NFTs, self-hosting an IPFS node means: your NFT metadata does not depend on whether Pinata's servers are online, does not depend on whether Infura's gateway is rate-limiting, does not depend on any third-party service -- as long as your node is online, your NFT data is reachable.

3. Solution #

Installing and Initializing IPFS Kubo #

# ====== 1. Install IPFS Kubo (formerly go-ipfs) ======
# Linux
wget https://dist.ipfs.tech/kubo/latest/kubo_linux-amd64.tar.gz
tar -xvzf kubo_linux-amd64.tar.gz
cd kubo
sudo bash install.sh

# macOS
brew install ipfs

# ====== 2. Initialize the node ======
ipfs init
# Creates the ~/.ipfs/ directory, containing:
#   config       — Node configuration file
#   datastore    — Data storage (default: flatfs)
#   keystore     — Node key and IPNS keys
#   blocks       — Content block storage

The config file generated by ipfs init contains the node's complete configuration, including:

• Addresses: Network addresses the node listens on (API, Gateway, Swarm)
• Bootstrap: List of initial bootstrap nodes to connect to
• Datastore: Storage configuration (StorageMax limit, GC policy)
• Identity: The node's PeerID and private key

Configuring the Node #

# ====== 3. Adjust storage limit (default is only 10 GB) ======
ipfs config Datastore.StorageMax 100GB

# ====== 4. Configure garbage collection (auto-clean unpinned content every hour) ======
ipfs config Datastore.GCPeriod 1h

# ====== 5. Enable public gateway (optional — allows anyone to access IPFS content through your node) ======
ipfs config --json Addresses.Gateway '"/ip4/0.0.0.0/tcp/8080"'

# ====== 6. View full configuration ======
ipfs config show

Starting the Daemon #

# Method 1: Foreground
ipfs daemon

# Method 2: systemd service (recommended for production)
sudo tee /etc/systemd/system/ipfs.service << 'EOF'
[Unit]
Description=IPFS Daemon
After=network.target

[Service]
Type=simple
User=ubuntu
ExecStart=/usr/local/bin/ipfs daemon
Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable ipfs
sudo systemctl start ipfs
sudo systemctl status ipfs

Adding and Pinning Content #

# ====== 7. Add a file ======
echo "ETH Tech Tree NFT Metadata" > metadata.json
ipfs add metadata.json
# Output: added QmXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXx metadata.json
#         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#         CID (Content Hash) — this is the file's permanent address

# ====== 8. Pin content (prevent it from being GC'd) ======
ipfs pin add QmXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXx

# ====== 9. Add an entire directory (NFT collection) ======
ipfs add -r my-nft-collection/
# Output: added QmYyYyYy... my-nft-collection
# The directory itself also has a CID (computed from the CIDs of all files within it)

# ====== 10. View pinned content ======
ipfs pin ls              # List all pins (local)
ipfs pin ls --type=recursive  # Show only recursive pins

# ====== 11. Access via local gateway ======
curl http://localhost:8080/ipfs/QmXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXx
# Returns: "ETH Tech Tree NFT Metadata"

Integration with Solidity NFT Contracts #

When deploying an NFT contract, tokenURI returns an IPFS URI:

// Use the ipfs:// protocol prefix in the contract
string memory uri = string(abi.encodePacked("ipfs://", cid));

// Or use an HTTP gateway as fallback (better marketplace/wallet compatibility)
string memory gatewayUri = string(abi.encodePacked(
    "https://ipfs.io/ipfs/", cid
));

// Or use your own node gateway (if you have a public-facing node)
string memory selfHostedUri = string(abi.encodePacked(
    "https://my-ipfs-node.example.com/ipfs/", cid
));

IPNS: Mutable Pointers to Immutable Content #

Since IPFS content is immutable (changing a file = new CID), you need a way to "point to" the latest version. IPNS (InterPlanetary Name System) provides a mutable naming system:

# ====== 12. Create an IPNS key pair (for your NFT project) ======
ipfs key gen my-nft-project
# Output: k51qzi5uqu5...

# ====== 13. Publish an IPNS record ======
ipfs name publish --key=my-nft-project QmNewVersionCID
# Now /ipns/k51qzi5uqu5... -> points to QmNewVersionCID

# ====== 14. Update the IPNS pointer (after updating NFT metadata) ======
ipfs add updated-metadata.json  # Get new CID
ipfs name publish --key=my-nft-project QmUpdatedCID

# Use IPNS in the contract:
# string memory uri = string(abi.encodePacked("ipns://k51qzi5uqu5..."));

Key IPNS property: after each publish update, all users accessing via ipns://key automatically receive the latest pointed-to CID -- no need to update the URI in the smart contract.

Complete NFT Project Workflow #

1. Generate NFT metadata JSON (name, description, attributes)
2. Generate NFT images (SVG or PNG)
3. ipfs add image.png -> obtain image CID
4. Reference "image": "ipfs://imageCID" in metadata.json
5. ipfs add metadata.json -> obtain metadata CID
6. Deploy smart contract; tokenURI() returns "ipfs://metadataCID"
7. Pin all content on your IPFS node (images + metadata)
8. Optional: Create an IPNS name pointing to the collection root directory
9. Optional: Redundantly pin on Pinata/web3.storage (double insurance)
10. Optional: Use Filecoin for decentralized long-term storage

4. Pitfalls Encountered #

• The "permanent storage" myth: IPFS is not Filecoin -- it has no built-in economic incentive mechanism to ensure content persistence. The content you pin exists only on the nodes that pin it -- if your node goes offline and no other node has pinned the same content, the content disappears from the network
• CID immutability trap: Modifying NFT metadata (even by one byte) means a completely new CID -- but the tokenURI in an already-deployed contract returns the old CID. Without IPNS or an upgradeable contract, you cannot update the metadata
• Garbage collection accidentally deleting content: Unpinned content is automatically cleaned up by GC -- if you ipfs add content but don't ipfs pin add it, it may no longer be on your node after 1 hour
• Poor public network reachability: New nodes need time to discover peers in the DHT network -- nodes behind NAT (home routers) may go days without being found by other nodes
• CID version confusion: IPFS has two CID formats -- CIDv0 (starting with Qm, Base58) and CIDv1 (starting with b, Base32) -- different tools and libraries default to different formats
• Public gateways do not guarantee service: https://ipfs.io/ipfs/CID is a centralized gateway -- ipfs.io may rate-limit, experience downtime, or change its terms of service

5. Why the Pitfalls Exist #

The misconception that "IPFS is permanent storage" stems from IPFS's underlying description -- "distributed file system." In reality, IPFS provides only content addressing and peer-to-peer transfer -- persistence requires an additional incentive layer (Filecoin) or active pinning. Analogy: IPFS is BitTorrent (peer-to-peer transfer), Filecoin is a seed-keeping service (economic incentive for persistence). Without pinning, your content is like a torrent with no seeders -- technically it exists, but it is unreachable.

CID immutability is the inevitable consequence of content addressing: CID = hash(content). Change the content, the hash changes, the CID must change. This is not a bug but a feature -- it guarantees content integrity (the content you request by CID is exactly the content you expect; it cannot be tampered with). However, this "immutability" becomes a disadvantage when you need to update metadata -- you need an additional mutable layer (IPNS, ENS, or an upgradeable contract's _baseURI) to redirect.

Poor connectivity for nodes behind NAT is because IPFS relies on a decentralized DHT (Kademlia DHT) for peer discovery. The DHT is bidirectional -- other nodes need to be able to connect directly to your node. If your router doesn't have port forwarding enabled (TCP 4001, default), you can only connect to discovered nodes, but other nodes cannot actively connect to you -- this means your node won't appear in DHT lookup results, and the CIDs you provide are unreachable to other nodes.

6. How to Resolve the Pitfalls #

Multi-layer redundant pinning strategy -- don't rely solely on your own node:

# Your own node (primary pin)
ipfs pin add QmCID

# Pinata (backup pinning service)
# https://pinata.cloud — free 1 GB, paid plans for more
curl -X POST "https://api.pinata.cloud/pinning/pinByHash" \
  -H "Authorization: Bearer $PINATA_JWT" \
  -d '{"hashToPin": "QmCID"}'

# web3.storage (Filecoin-backed backup)
# https://web3.storage — free 5 GB, auto-backup to Filecoin

Use IPNS for mutable metadata:

# Generate IPNS key at project initialization
ipfs key gen my-nft

# Use ipns:// instead of ipfs:// when deploying the contract
# tokenURI = "ipns://k51qzi5uqu5..." 
# Ethereum libraries need special handling for IPNS; typically use HTTP gateway proxying

# Practical approach: use an upgradeable _baseURI
# In the NFT contract:
string public baseURI;  // modifiable by owner

function setBaseURI(string memory _newBaseURI) external onlyOwner {
    baseURI = _newBaseURI;  // Point to new IPFS CID
}

Ensure GC doesn't delete critical data: Pin immediately after adding content:

# One-step add and pin (using a shell function)
add-and-pin() {
    local cid=$(ipfs add -Q "$1")
    ipfs pin add "$cid"
    echo "Added and pinned: $cid"
}

add-and-pin metadata.json

The -Q flag makes ipfs add output only the CID (no filename), convenient for chaining.

Resolve NAT issues:

# 1. Forward port 4001 (TCP+UDP) to your node on your router
# 2. Configure IPFS to use the public address
ipfs config --json Addresses.Swarm '[
    "/ip4/0.0.0.0/tcp/4001",
    "/ip6/::/tcp/4001"
]'

# 3. Manually advertise public IP (if UPnP is unavailable)
ipfs config --json Routing.AcceleratedDHTClient true

# 4. Verify connectivity
ipfs swarm peers        # View number of connected peers
ipfs id                 # View your PeerID and addresses

For CID version compatibility, maintain consistency: for NFT scenarios, CIDv0 (Qm...) is recommended because it has the broadest compatibility -- most wallet and marketplace tools default to CIDv0. Explicitly convert when CIDv1 features (multi-hash algorithms, multi-codecs) are needed:

# Convert CIDv0 to CIDv1
ipfs cid base32 QmXxXxXx...
# Output: bafybei...

7. Key Technical Points #