Turborepo & PNPM Migration

This document outlines the recent migration from NPM to PNPM and the implementation of Turborepo for monorepo build orchestration.

Overview

We migrated the Earna AI monorepo from NPM workspaces to PNPM workspaces and implemented Turborepo for efficient build orchestration. This migration resolved critical native dependency issues and significantly improved build performance.

Key Changes

1. Package Manager Migration (NPM → PNPM)

Why PNPM?

• Native dependency resolution: Resolved issues with packages like lightningcss and @napi-rs/simple-git
• Disk space efficiency: Uses hard links instead of copying node_modules
• Faster installations: Parallel installation and better caching
• Strict dependency resolution: Prevents phantom dependencies

Migration Steps

1. Removed package-lock.json files from all projects
2. Created pnpm-workspace.yaml to define workspace structure
3. Added .npmrc with node-linker=hoisted for compatibility
4. Generated pnpm-lock.yaml for reproducible builds

2. Turborepo Implementation

Configuration (turbo.json)

{
  "$schema": "https://turbo.build/schema.json",
  "tasks": {
    "build": {
      "dependsOn": ["^build"],
      "outputs": [".next/**", "!.next/cache/**", "dist/**"],
      "env": [
        "NODE_ENV",
        "NEXT_PUBLIC_*",
        "DATABASE_URL",
        "PNPM_VERSION",
        "VERCEL_DEEP_CLONE"
      ]
    },
    "dev": {
      "cache": false,
      "persistent": true
    },
    "lint": {
      "dependsOn": ["^build"]
    },
    "typecheck": {
      "dependsOn": ["^build"]
    }
  }
}

Benefits

• Incremental builds: Only rebuilds what changed
• Parallel execution: Runs tasks across workspaces simultaneously
• Smart caching: Caches build outputs for faster subsequent builds
• Pipeline orchestration: Ensures correct build order with dependsOn

3. Workspace Structure

# pnpm-workspace.yaml
packages:
  - 'apps'
  - 'console'
  - 'docs-nextra'
  - 'credit-engine'

Each workspace is an independent Next.js application with its own:

• package.json for dependencies
• vercel.json for deployment configuration
• .next/ build output directory

4. Vercel Deployment Configuration

Each project has updated vercel.json:

{
  "$schema": "https://openapi.vercel.sh/vercel.json",
  "framework": "nextjs",
  "buildCommand": "pnpm turbo build --filter=[project-name]",
  "installCommand": "pnpm install",
  "outputDirectory": ".next"
}

Fixed Issues

1. Native Dependency Errors

Problem: lightningcss and @napi-rs/simple-git failed with NPM Solution: PNPM’s better native module handling resolved these issues

2. D3.js Import Errors

Problem: Wildcard imports from d3 failed in production Solution: Migrated to specific sub-package imports:

// Before (broken)
import * as d3 from "d3"
 
// After (working)
import { max, range } from "d3-array"
import { geoAlbers } from "d3-geo"
import { scaleLinear, scaleSequential } from "d3-scale"

3. TypeScript Animation Errors

Problem: String literals not assignable to animation types Solution: Used as const assertions:

const TRANSITION = {
  type: "spring" as const,
  duration: 0.2,
  bounce: 0,
}

4. Preview Deployment 404s

Problem: Console showed 404 in preview branches Root Cause: Conflicting root directory configuration Solution: Aligned build commands with project structure

Commands Reference

Development

# Install dependencies
pnpm install
 
# Run development server for specific app
pnpm --filter console dev
pnpm --filter apps dev
pnpm --filter docs-nextra dev
 
# Run all dev servers
pnpm turbo dev

Building

# Build specific app
pnpm turbo build --filter=console
pnpm turbo build --filter=apps
pnpm turbo build --filter=docs-nextra
 
# Build all apps
pnpm turbo build

Linting & Type Checking

# Lint specific app
pnpm turbo lint --filter=console
 
# Type check all apps
pnpm turbo typecheck

Environment Variables

Required environment variables for builds:

• NODE_ENV: Production/development environment
• NEXT_PUBLIC_*: All Next.js public environment variables
• DATABASE_URL: Database connection string
• PNPM_VERSION: PNPM version for Vercel builds
• VERCEL_DEEP_CLONE: Enable deep cloning for monorepo

Performance Improvements

• Build time: ~60% reduction with Turborepo caching
• Install time: ~40% faster with PNPM
• Disk usage: ~30% reduction with hard links
• CI/CD: Parallel builds across all projects

Troubleshooting

PNPM Installation Issues

If you encounter installation issues:

# Clear PNPM cache
pnpm store prune
 
# Reinstall dependencies
rm -rf node_modules pnpm-lock.yaml
pnpm install

Turborepo Cache Issues

To clear Turborepo cache:

# Clear all cache
rm -rf .turbo
 
# Clear specific task cache
pnpm turbo build --force

Vercel Deployment Issues

Ensure these settings in Vercel:

1. Install Command: pnpm install
2. Build Command: pnpm turbo build --filter=[project-name]
3. Output Directory: .next
4. Root Directory: Project root (not subdirectory)

Migration Checklist

• Remove all package-lock.json files
• Create pnpm-workspace.yaml
• Add .npmrc configuration
• Create turbo.json configuration
• Update all vercel.json files
• Fix D3.js imports to use sub-packages
• Add missing TypeScript type definitions
• Update CI/CD workflows
• Test all preview deployments
• Verify production deployments

Next Steps

1. Monitor build performance metrics
2. Optimize Turborepo cache strategy
3. Consider remote caching for team collaboration
4. Implement stricter dependency management policies