bibdle/plans/local-bible-xml-implementation.md

Local Bible XML Implementation Plan

Overview

Replace the external Bible API (fetchRandomVerse()) with a local XML-based solution using the downloaded NKJV Bible XML file.

Current Implementation Analysis

Current Flow

The system currently:

1. Calls fetchRandomVerse() which hits https://bible-api.com
2. Gets a random verse position (book, chapter, verse)
3. Fetches 3 consecutive verses starting from that position
4. Returns { bookId, reference, verseText } as ApiVerse
5. Stores this in the database via getTodayVerse()

Key Data Structures

• Book IDs: 3-letter codes (e.g., 'GEN', 'MAT', '1CO') defined in bibleBooks
• Book Order: Sequential 1-66 via the order property
• Return Format: { bookId: string, reference: string, verseText: string }

XML Structure

<bible translation="English NKJ 1982">
  <testament name="Old">
    <book number="1">
      <chapter number="1">
        <verse number="1">Text here</verse>

Books are numbered 1-66, matching the order property in bibleBooks.

Solution Architecture

Book Number Mapping Strategy

Since the XML uses numbers (1-66) and the system uses book IDs ('GEN', 'MAT', etc.), we'll create a mapping using the existing bibleBooks array:

// book number -> book ID
const bookNumberToId = {
  1: "GEN", // Genesis
  2: "EXO", // Exodus
  // ... through ...
  40: "MAT", // Matthew (first NT book)
  // ... to ...
  66: "REV", // Revelation
};

This can be generated programmatically from the existing bibleBooks array using the order property.

XML Parsing Approach

We'll use a streaming/lazy approach to avoid loading the entire XML into memory:

1. Random Selection: Pick random book number (1-66), chapter, and verse
2. Targeted Parsing: Parse only the needed section of the XML
3. Verse Extraction: Extract 3 consecutive verses from that position

Implementation Flow

graph TD
    A[getTodayVerse called] --> B[fetchRandomVerse]
    B --> C[Generate random position]
    C --> D[Load XML file]
    D --> E[Parse specific book/chapter]
    E --> F[Extract 3 consecutive verses]
    F --> G[Map book number to ID]
    G --> H[Format reference string]
    H --> I[Return ApiVerse object]
    I --> J[Store in database]

Technical Implementation Plan

Step 1: Create Book Mapping Utility

File: src/lib/server/bible.ts

Add a new mapping object and helper function:

• Create bookNumberToId map derived from bibleBooks
• Create getBookByNumber(number: number): BibleBook | undefined
• Create reverse map bookIdToNumber for potential future use

Step 2: Install XML Parser

Command: Install a lightweight XML parser

Options:

• fast-xml-parser - Fast, lightweight, good for selective parsing
• xml2js - Popular, stable
• Node's built-in stream parser for maximum efficiency

Recommendation: fast-xml-parser for balance of speed and ease of use.

Step 3: Create XML Utility Module

File: src/lib/server/xml-bible.ts (new file)

Functions needed:

• loadBibleXml(): Read the XML file from disk
• parseBook(xml: string, bookNumber: number): Extract specific book data
• getChapterVerseCount(bookNumber: number, chapterNumber: number): Get verse count for validation
• extractVerses(bookNumber: number, chapter: number, startVerse: number, count: number): Get consecutive verses

Step 4: Implement Random Verse Selection Logic

File: src/lib/server/xml-bible.ts

Algorithm:

1. Pick random book number (1-66)
2. Parse that book to get chapter count
3. Pick random chapter
4. Parse that chapter to get verse count
5. Pick random starting verse (ensuring room for 3 verses)
6. Extract verses and verse count

Optimization: Consider pre-computing chapter/verse counts to avoid parsing overhead.

Step 5: Create New fetchRandomVerse Implementation

File: src/lib/server/bible-api.ts

Replace the current implementation:

• Remove external API calls
• Use new XML parsing functions
• Maintain the same return type ApiVerse
• Format reference string (e.g., "Matthew 1:1-3")

Step 6: Handle Edge Cases

Important considerations:

• End-of-chapter: If starting verse is near end, may get fewer than 3 verses
• End-of-book: Don't cross chapter boundaries
• Short chapters: Some chapters have only 1-2 verses
• Book names: Generate proper reference strings (e.g., "1 Samuel" not "1SA")

Step 7: Testing Strategy

1. Manual testing: Run the app and verify verses load correctly
2. Verify book mapping: Ensure all 66 books map correctly
3. Check reference format: Ensure references match expected format
4. Edge case testing: Test with short chapters/books
5. Performance: Ensure parsing is fast enough for production

Data Structure Details

Book Number to ID Mapping

// Complete mapping based on existing bibleBooks array
const BOOK_MAPPING = {
  // Old Testament (1-39)
  1: "GEN",
  2: "EXO",
  3: "LEV",
  4: "NUM",
  5: "DEU",
  6: "JOS",
  7: "JDG",
  8: "RUT",
  9: "1SA",
  10: "2SA",
  11: "1KI",
  12: "2KI",
  13: "1CH",
  14: "2CH",
  15: "EZR",
  16: "NEH",
  17: "EST",
  18: "JOB",
  19: "PSA",
  20: "PRO",
  21: "ECC",
  22: "SNG",
  23: "ISA",
  24: "JER",
  25: "LAM",
  26: "EZK",
  27: "DAN",
  28: "HOS",
  29: "JOL",
  30: "AMO",
  31: "OBA",
  32: "JON",
  33: "MIC",
  34: "NAM",
  35: "HAB",
  36: "ZEP",
  37: "HAG",
  38: "ZEC",
  39: "MAL",
  // New Testament (40-66)
  40: "MAT",
  41: "MRK",
  42: "LUK",
  43: "JHN",
  44: "ACT",
  45: "ROM",
  46: "1CO",
  47: "2CO",
  48: "GAL",
  49: "EPH",
  50: "PHP",
  51: "COL",
  52: "1TH",
  53: "2TH",
  54: "1TI",
  55: "2TI",
  56: "TIT",
  57: "PHM",
  58: "HEB",
  59: "JAS",
  60: "1PE",
  61: "2PE",
  62: "1JN",
  63: "2JN",
  64: "3JN",
  65: "JUD",
  66: "REV",
};

Reference String Format

The reference should match the current format from the API:

• Format: "{Book Name} {Chapter}:{StartVerse}-{EndVerse}"
• Example: "Matthew 1:1-3" or "1 Corinthians 13:4-6"

Use the name property from the BibleBook interface.

File Changes Summary

New Files

• src/lib/server/xml-bible.ts - XML parsing and verse extraction logic

Modified Files

• src/lib/server/bible.ts - Add book number mapping utilities
• src/lib/server/bible-api.ts - Replace API calls with local XML parsing
• package.json - Add XML parser dependency

No Changes Required

• src/routes/+page.server.ts - Uses fetchRandomVerse(), internal implementation change is transparent
• src/lib/types/bible.ts - Existing types remain the same
• Database schema - No changes needed

Performance Considerations

Memory Usage

• Avoid loading entire XML file into memory
• Parse only the needed book/chapter sections
• Use streaming or selective parsing

Speed Optimization Options

1. Lazy parsing: Only parse when needed (slower, low memory)
2. Pre-parsed index: Create JSON index of book/chapter/verse counts (faster, more memory)
3. Caching: Cache parsed books in memory during runtime (fastest, higher memory)

Recommendation: Start with lazy parsing, optimize if needed.

File Size

The XML file is large but will be read from disk once per verse generation. Since verses are cached per day in the database, this happens once daily.

Rollback Strategy

If issues arise:

1. Keep the old fetchRandomVerse() implementation as fetchRandomVerseAPI()
2. Add environment variable to toggle between implementations
3. Can quickly revert by changing which function is exported

Next Steps After Implementation

1. Monitor daily verse generation for errors
2. Verify reference strings are formatted correctly
3. Consider adding verse length validation (avoid very short/long passages)
4. Potentially add weighted random selection to favor popular books

Questions to Resolve

1. Should we validate that selected verses have minimum text length?
2. Should we add any filtering to avoid certain books or chapters?
3. Do we want to pre-compute chapter/verse counts for faster selection?