switched to NKJV, improved grading, improved styling

2026-02-04 10:54:44 -05:00 · 2025-12-23 17:33:33 -05:00
parent 93acafc232
commit f9f0928278
16 changed files with 34345 additions and 68 deletions
--- a/plans/local-bible-xml-implementation.md
+++ b/plans/local-bible-xml-implementation.md
@@ -0,0 +1,302 @@
+# Local Bible XML Implementation Plan
+
+## Overview
+
+Replace the external Bible API ([`fetchRandomVerse()`](../src/lib/server/bible-api.ts:6)) with a local XML-based solution using the downloaded NKJV Bible XML file.
+
+## Current Implementation Analysis
+
+### Current Flow
+
+The system currently:
+
+1. Calls [`fetchRandomVerse()`](../src/lib/server/bible-api.ts:6) which hits `https://bible-api.com`
+2. Gets a random verse position (book, chapter, verse)
+3. Fetches 3 consecutive verses starting from that position
+4. Returns `{ bookId, reference, verseText }` as [`ApiVerse`](../src/lib/server/bible-api.ts:4)
+5. Stores this in the database via [`getTodayVerse()`](../src/routes/+page.server.ts:11)
+
+### Key Data Structures
+
+- **Book IDs**: 3-letter codes (e.g., 'GEN', 'MAT', '1CO') defined in [`bibleBooks`](../src/lib/types/bible.ts:24)
+- **Book Order**: Sequential 1-66 via the `order` property
+- **Return Format**: `{ bookId: string, reference: string, verseText: string }`
+
+### XML Structure
+
+```xml
+<bible translation="English NKJ 1982">
+  <testament name="Old">
+    <book number="1">
+      <chapter number="1">
+        <verse number="1">Text here</verse>
+```
+
+Books are numbered 1-66, matching the `order` property in [`bibleBooks`](../src/lib/types/bible.ts:24).
+
+## Solution Architecture
+
+### Book Number Mapping Strategy
+
+Since the XML uses numbers (1-66) and the system uses book IDs ('GEN', 'MAT', etc.), we'll create a mapping using the existing [`bibleBooks`](../src/lib/types/bible.ts:24) array:
+
+```typescript
+// book number -> book ID
+const bookNumberToId = {
+  1: "GEN", // Genesis
+  2: "EXO", // Exodus
+  // ... through ...
+  40: "MAT", // Matthew (first NT book)
+  // ... to ...
+  66: "REV", // Revelation
+};
+```
+
+This can be generated programmatically from the existing [`bibleBooks`](../src/lib/types/bible.ts:24) array using the `order` property.
+
+### XML Parsing Approach
+
+We'll use a streaming/lazy approach to avoid loading the entire XML into memory:
+
+1. **Random Selection**: Pick random book number (1-66), chapter, and verse
+2. **Targeted Parsing**: Parse only the needed section of the XML
+3. **Verse Extraction**: Extract 3 consecutive verses from that position
+
+### Implementation Flow
+
+```mermaid
+graph TD
+    A[getTodayVerse called] --> B[fetchRandomVerse]
+    B --> C[Generate random position]
+    C --> D[Load XML file]
+    D --> E[Parse specific book/chapter]
+    E --> F[Extract 3 consecutive verses]
+    F --> G[Map book number to ID]
+    G --> H[Format reference string]
+    H --> I[Return ApiVerse object]
+    I --> J[Store in database]
+```
+
+## Technical Implementation Plan
+
+### Step 1: Create Book Mapping Utility
+
+**File**: [`src/lib/server/bible.ts`](../src/lib/server/bible.ts)
+
+Add a new mapping object and helper function:
+
+- Create `bookNumberToId` map derived from [`bibleBooks`](../src/lib/types/bible.ts:24)
+- Create `getBookByNumber(number: number): BibleBook | undefined`
+- Create reverse map `bookIdToNumber` for potential future use
+
+### Step 2: Install XML Parser
+
+**Command**: Install a lightweight XML parser
+
+Options:
+
+- `fast-xml-parser` - Fast, lightweight, good for selective parsing
+- `xml2js` - Popular, stable
+- Node's built-in stream parser for maximum efficiency
+
+Recommendation: `fast-xml-parser` for balance of speed and ease of use.
+
+### Step 3: Create XML Utility Module
+
+**File**: `src/lib/server/xml-bible.ts` (new file)
+
+Functions needed:
+
+- `loadBibleXml()`: Read the XML file from disk
+- `parseBook(xml: string, bookNumber: number)`: Extract specific book data
+- `getChapterVerseCount(bookNumber: number, chapterNumber: number)`: Get verse count for validation
+- `extractVerses(bookNumber: number, chapter: number, startVerse: number, count: number)`: Get consecutive verses
+
+### Step 4: Implement Random Verse Selection Logic
+
+**File**: `src/lib/server/xml-bible.ts`
+
+Algorithm:
+
+1. Pick random book number (1-66)
+2. Parse that book to get chapter count
+3. Pick random chapter
+4. Parse that chapter to get verse count
+5. Pick random starting verse (ensuring room for 3 verses)
+6. Extract verses and verse count
+
+**Optimization**: Consider pre-computing chapter/verse counts to avoid parsing overhead.
+
+### Step 5: Create New fetchRandomVerse Implementation
+
+**File**: [`src/lib/server/bible-api.ts`](../src/lib/server/bible-api.ts)
+
+Replace the current implementation:
+
+- Remove external API calls
+- Use new XML parsing functions
+- Maintain the same return type [`ApiVerse`](../src/lib/server/bible-api.ts:4)
+- Format reference string (e.g., "Matthew 1:1-3")
+
+### Step 6: Handle Edge Cases
+
+Important considerations:
+
+- **End-of-chapter**: If starting verse is near end, may get fewer than 3 verses
+- **End-of-book**: Don't cross chapter boundaries
+- **Short chapters**: Some chapters have only 1-2 verses
+- **Book names**: Generate proper reference strings (e.g., "1 Samuel" not "1SA")
+
+### Step 7: Testing Strategy
+
+1. **Manual testing**: Run the app and verify verses load correctly
+2. **Verify book mapping**: Ensure all 66 books map correctly
+3. **Check reference format**: Ensure references match expected format
+4. **Edge case testing**: Test with short chapters/books
+5. **Performance**: Ensure parsing is fast enough for production
+
+## Data Structure Details
+
+### Book Number to ID Mapping
+
+```typescript
+// Complete mapping based on existing bibleBooks array
+const BOOK_MAPPING = {
+  // Old Testament (1-39)
+  1: "GEN",
+  2: "EXO",
+  3: "LEV",
+  4: "NUM",
+  5: "DEU",
+  6: "JOS",
+  7: "JDG",
+  8: "RUT",
+  9: "1SA",
+  10: "2SA",
+  11: "1KI",
+  12: "2KI",
+  13: "1CH",
+  14: "2CH",
+  15: "EZR",
+  16: "NEH",
+  17: "EST",
+  18: "JOB",
+  19: "PSA",
+  20: "PRO",
+  21: "ECC",
+  22: "SNG",
+  23: "ISA",
+  24: "JER",
+  25: "LAM",
+  26: "EZK",
+  27: "DAN",
+  28: "HOS",
+  29: "JOL",
+  30: "AMO",
+  31: "OBA",
+  32: "JON",
+  33: "MIC",
+  34: "NAM",
+  35: "HAB",
+  36: "ZEP",
+  37: "HAG",
+  38: "ZEC",
+  39: "MAL",
+  // New Testament (40-66)
+  40: "MAT",
+  41: "MRK",
+  42: "LUK",
+  43: "JHN",
+  44: "ACT",
+  45: "ROM",
+  46: "1CO",
+  47: "2CO",
+  48: "GAL",
+  49: "EPH",
+  50: "PHP",
+  51: "COL",
+  52: "1TH",
+  53: "2TH",
+  54: "1TI",
+  55: "2TI",
+  56: "TIT",
+  57: "PHM",
+  58: "HEB",
+  59: "JAS",
+  60: "1PE",
+  61: "2PE",
+  62: "1JN",
+  63: "2JN",
+  64: "3JN",
+  65: "JUD",
+  66: "REV",
+};
+```
+
+### Reference String Format
+
+The reference should match the current format from the API:
+
+- Format: `"{Book Name} {Chapter}:{StartVerse}-{EndVerse}"`
+- Example: `"Matthew 1:1-3"` or `"1 Corinthians 13:4-6"`
+
+Use the `name` property from the [`BibleBook`](../src/lib/types/bible.ts:14) interface.
+
+## File Changes Summary
+
+### New Files
+
+- `src/lib/server/xml-bible.ts` - XML parsing and verse extraction logic
+
+### Modified Files
+
+- [`src/lib/server/bible.ts`](../src/lib/server/bible.ts) - Add book number mapping utilities
+- [`src/lib/server/bible-api.ts`](../src/lib/server/bible-api.ts) - Replace API calls with local XML parsing
+- `package.json` - Add XML parser dependency
+
+### No Changes Required
+
+- [`src/routes/+page.server.ts`](../src/routes/+page.server.ts) - Uses [`fetchRandomVerse()`](../src/lib/server/bible-api.ts:6), internal implementation change is transparent
+- [`src/lib/types/bible.ts`](../src/lib/types/bible.ts) - Existing types remain the same
+- Database schema - No changes needed
+
+## Performance Considerations
+
+### Memory Usage
+
+- Avoid loading entire XML file into memory
+- Parse only the needed book/chapter sections
+- Use streaming or selective parsing
+
+### Speed Optimization Options
+
+1. **Lazy parsing**: Only parse when needed (slower, low memory)
+2. **Pre-parsed index**: Create JSON index of book/chapter/verse counts (faster, more memory)
+3. **Caching**: Cache parsed books in memory during runtime (fastest, higher memory)
+
+Recommendation: Start with lazy parsing, optimize if needed.
+
+### File Size
+
+The XML file is large but will be read from disk once per verse generation. Since verses are cached per day in the database, this happens once daily.
+
+## Rollback Strategy
+
+If issues arise:
+
+1. Keep the old [`fetchRandomVerse()`](../src/lib/server/bible-api.ts:6) implementation as `fetchRandomVerseAPI()`
+2. Add environment variable to toggle between implementations
+3. Can quickly revert by changing which function is exported
+
+## Next Steps After Implementation
+
+1. Monitor daily verse generation for errors
+2. Verify reference strings are formatted correctly
+3. Consider adding verse length validation (avoid very short/long passages)
+4. Potentially add weighted random selection to favor popular books
+
+## Questions to Resolve
+
+1. Should we validate that selected verses have minimum text length?
+2. Should we add any filtering to avoid certain books or chapters?
+3. Do we want to pre-compute chapter/verse counts for faster selection?