bibdle/plans/local-bible-xml-implementation.md

# Local Bible XML Implementation Plan

## Overview

Replace the external Bible API ([`fetchRandomVerse()`](../src/lib/server/bible-api.ts:6)) with a local XML-based solution using the downloaded NKJV Bible XML file.

## Current Implementation Analysis

### Current Flow

The system currently:

1. Calls [`fetchRandomVerse()`](../src/lib/server/bible-api.ts:6) which hits `https://bible-api.com`
2. Gets a random verse position (book, chapter, verse)
3. Fetches 3 consecutive verses starting from that position
4. Returns `{ bookId, reference, verseText }` as [`ApiVerse`](../src/lib/server/bible-api.ts:4)
5. Stores this in the database via [`getTodayVerse()`](../src/routes/+page.server.ts:11)

### Key Data Structures

- **Book IDs**: 3-letter codes (e.g., 'GEN', 'MAT', '1CO') defined in [`bibleBooks`](../src/lib/types/bible.ts:24)
- **Book Order**: Sequential 1-66 via the `order` property
- **Return Format**: `{ bookId: string, reference: string, verseText: string }`

### XML Structure

```xml
<bible translation="English NKJ 1982">
  <testament name="Old">
    <book number="1">
      <chapter number="1">
        <verse number="1">Text here</verse>
```

Books are numbered 1-66, matching the `order` property in [`bibleBooks`](../src/lib/types/bible.ts:24).

## Solution Architecture

### Book Number Mapping Strategy

Since the XML uses numbers (1-66) and the system uses book IDs ('GEN', 'MAT', etc.), we'll create a mapping using the existing [`bibleBooks`](../src/lib/types/bible.ts:24) array:

```typescript
// book number -> book ID
const bookNumberToId = {
  1: "GEN", // Genesis
  2: "EXO", // Exodus
  // ... through ...
  40: "MAT", // Matthew (first NT book)
  // ... to ...
  66: "REV", // Revelation
};
```

This can be generated programmatically from the existing [`bibleBooks`](../src/lib/types/bible.ts:24) array using the `order` property.

### XML Parsing Approach

We'll use a streaming/lazy approach to avoid loading the entire XML into memory:

1. **Random Selection**: Pick random book number (1-66), chapter, and verse
2. **Targeted Parsing**: Parse only the needed section of the XML
3. **Verse Extraction**: Extract 3 consecutive verses from that position

### Implementation Flow

```mermaid
graph TD
    A[getTodayVerse called] --> B[fetchRandomVerse]
    B --> C[Generate random position]
    C --> D[Load XML file]
    D --> E[Parse specific book/chapter]
    E --> F[Extract 3 consecutive verses]
    F --> G[Map book number to ID]
    G --> H[Format reference string]
    H --> I[Return ApiVerse object]
    I --> J[Store in database]
```

## Technical Implementation Plan

### Step 1: Create Book Mapping Utility

**File**: [`src/lib/server/bible.ts`](../src/lib/server/bible.ts)

Add a new mapping object and helper function:

- Create `bookNumberToId` map derived from [`bibleBooks`](../src/lib/types/bible.ts:24)
- Create `getBookByNumber(number: number): BibleBook | undefined`
- Create reverse map `bookIdToNumber` for potential future use

### Step 2: Install XML Parser

**Command**: Install a lightweight XML parser

Options:

- `fast-xml-parser` - Fast, lightweight, good for selective parsing
- `xml2js` - Popular, stable
- Node's built-in stream parser for maximum efficiency

Recommendation: `fast-xml-parser` for balance of speed and ease of use.

### Step 3: Create XML Utility Module

**File**: `src/lib/server/xml-bible.ts` (new file)

Functions needed:

- `loadBibleXml()`: Read the XML file from disk
- `parseBook(xml: string, bookNumber: number)`: Extract specific book data
- `getChapterVerseCount(bookNumber: number, chapterNumber: number)`: Get verse count for validation
- `extractVerses(bookNumber: number, chapter: number, startVerse: number, count: number)`: Get consecutive verses

### Step 4: Implement Random Verse Selection Logic

**File**: `src/lib/server/xml-bible.ts`

Algorithm:

1. Pick random book number (1-66)
2. Parse that book to get chapter count
3. Pick random chapter
4. Parse that chapter to get verse count
5. Pick random starting verse (ensuring room for 3 verses)
6. Extract verses and verse count

**Optimization**: Consider pre-computing chapter/verse counts to avoid parsing overhead.

### Step 5: Create New fetchRandomVerse Implementation

**File**: [`src/lib/server/bible-api.ts`](../src/lib/server/bible-api.ts)

Replace the current implementation:

- Remove external API calls
- Use new XML parsing functions
- Maintain the same return type [`ApiVerse`](../src/lib/server/bible-api.ts:4)
- Format reference string (e.g., "Matthew 1:1-3")

### Step 6: Handle Edge Cases

Important considerations:

- **End-of-chapter**: If starting verse is near end, may get fewer than 3 verses
- **End-of-book**: Don't cross chapter boundaries
- **Short chapters**: Some chapters have only 1-2 verses
- **Book names**: Generate proper reference strings (e.g., "1 Samuel" not "1SA")

### Step 7: Testing Strategy

1. **Manual testing**: Run the app and verify verses load correctly
2. **Verify book mapping**: Ensure all 66 books map correctly
3. **Check reference format**: Ensure references match expected format
4. **Edge case testing**: Test with short chapters/books
5. **Performance**: Ensure parsing is fast enough for production

## Data Structure Details

### Book Number to ID Mapping

```typescript
// Complete mapping based on existing bibleBooks array
const BOOK_MAPPING = {
  // Old Testament (1-39)
  1: "GEN",
  2: "EXO",
  3: "LEV",
  4: "NUM",
  5: "DEU",
  6: "JOS",
  7: "JDG",
  8: "RUT",
  9: "1SA",
  10: "2SA",
  11: "1KI",
  12: "2KI",
  13: "1CH",
  14: "2CH",
  15: "EZR",
  16: "NEH",
  17: "EST",
  18: "JOB",
  19: "PSA",
  20: "PRO",
  21: "ECC",
  22: "SNG",
  23: "ISA",
  24: "JER",
  25: "LAM",
  26: "EZK",
  27: "DAN",
  28: "HOS",
  29: "JOL",
  30: "AMO",
  31: "OBA",
  32: "JON",
  33: "MIC",
  34: "NAM",
  35: "HAB",
  36: "ZEP",
  37: "HAG",
  38: "ZEC",
  39: "MAL",
  // New Testament (40-66)
  40: "MAT",
  41: "MRK",
  42: "LUK",
  43: "JHN",
  44: "ACT",
  45: "ROM",
  46: "1CO",
  47: "2CO",
  48: "GAL",
  49: "EPH",
  50: "PHP",
  51: "COL",
  52: "1TH",
  53: "2TH",
  54: "1TI",
  55: "2TI",
  56: "TIT",
  57: "PHM",
  58: "HEB",
  59: "JAS",
  60: "1PE",
  61: "2PE",
  62: "1JN",
  63: "2JN",
  64: "3JN",
  65: "JUD",
  66: "REV",
};
```

### Reference String Format

The reference should match the current format from the API:

- Format: `"{Book Name} {Chapter}:{StartVerse}-{EndVerse}"`
- Example: `"Matthew 1:1-3"` or `"1 Corinthians 13:4-6"`

Use the `name` property from the [`BibleBook`](../src/lib/types/bible.ts:14) interface.

## File Changes Summary

### New Files

- `src/lib/server/xml-bible.ts` - XML parsing and verse extraction logic

### Modified Files

- [`src/lib/server/bible.ts`](../src/lib/server/bible.ts) - Add book number mapping utilities
- [`src/lib/server/bible-api.ts`](../src/lib/server/bible-api.ts) - Replace API calls with local XML parsing
- `package.json` - Add XML parser dependency

### No Changes Required

- [`src/routes/+page.server.ts`](../src/routes/+page.server.ts) - Uses [`fetchRandomVerse()`](../src/lib/server/bible-api.ts:6), internal implementation change is transparent
- [`src/lib/types/bible.ts`](../src/lib/types/bible.ts) - Existing types remain the same
- Database schema - No changes needed

## Performance Considerations

### Memory Usage

- Avoid loading entire XML file into memory
- Parse only the needed book/chapter sections
- Use streaming or selective parsing

### Speed Optimization Options

1. **Lazy parsing**: Only parse when needed (slower, low memory)
2. **Pre-parsed index**: Create JSON index of book/chapter/verse counts (faster, more memory)
3. **Caching**: Cache parsed books in memory during runtime (fastest, higher memory)

Recommendation: Start with lazy parsing, optimize if needed.

### File Size

The XML file is large but will be read from disk once per verse generation. Since verses are cached per day in the database, this happens once daily.

## Rollback Strategy

If issues arise:

1. Keep the old [`fetchRandomVerse()`](../src/lib/server/bible-api.ts:6) implementation as `fetchRandomVerseAPI()`
2. Add environment variable to toggle between implementations
3. Can quickly revert by changing which function is exported

## Next Steps After Implementation

1. Monitor daily verse generation for errors
2. Verify reference strings are formatted correctly
3. Consider adding verse length validation (avoid very short/long passages)
4. Potentially add weighted random selection to favor popular books

## Questions to Resolve

1. Should we validate that selected verses have minimum text length?
2. Should we add any filtering to avoid certain books or chapters?
3. Do we want to pre-compute chapter/verse counts for faster selection?