A language model can summarize a note beautifully and still answer the wrong question. Ask, “What did we decide about the October launch?” and an ungrounded model may produce a plausible plan from general knowledge. A useful notes assistant must answer from the notes the user owns.

That is retrieval-augmented generation, or RAG: retrieve relevant records, then give them to a model as context. On iOS 27, SpotlightSearchTool connects a Core Spotlight index to LanguageModelSession. This article builds that flow while keeping the database authoritative, the index recoverable, and the returned notes visible as sources.

Availability: Apple’s natural-language Core Spotlight sample requires iOS 27 or later, Apple Intelligence support, and Apple Intelligence enabled. Check the SDK annotations for every platform you ship.

1. Define the Trust Boundary

Core Spotlight is a derived search surface, not a second database. A request follows this path:

  1. The model calls SpotlightSearchTool for the user’s question.
  2. Spotlight retrieves candidates from selected indexed attributes.
  3. An index delegate hydrates complete records when required.
  4. The model generates an answer from that context.
  5. The app displays returned records independently as sources.

Generated prose is a synthesis. Source cards are inspectable evidence. If your app already hides models behind a service, keep that boundary; the provider-agnostic Foundation Models architecture shows how to keep model construction out of views.

2. Index Stable, Useful Metadata

Retrieval quality begins with the index. Use the persistence identifier for Spotlight identity, add fields with clear meaning, and include a deep link.

import CoreSpotlight
import UniformTypeIdentifiers

struct NoteSnapshot: Sendable {
    let id: UUID
    let title: String
    let body: String
    let tags: [String]
    let modifiedAt: Date
}

enum NoteSearchItemFactory {
    static func makeItem(_ note: NoteSnapshot) -> CSSearchableItem {
        let attributes = CSSearchableItemAttributeSet(contentType: .text)
        attributes.title = note.title
        attributes.textContent = note.body
        attributes.keywords = note.tags
        attributes.contentModificationDate = note.modifiedAt
        attributes.contentURL = URL(
            string: "warmonotes://note/\(note.id.uuidString)"
        )
        return CSSearchableItem(
            uniqueIdentifier: note.id.uuidString,
            domainIdentifier: "notes",
            attributeSet: attributes
        )
    }
}

Index only content the feature needs. Passwords, recovery keys, hidden access-control state, and sensitive previews do not belong in broadly useful metadata. Durable identity also helps when notes appear through App Intents in Siri, Spotlight, and Shortcuts.

Make reconciliation crash-safe

An index write and a database transaction cannot commit atomically. The process can die after Spotlight accepts an item but before your manifest records it. If the note is then deleted, manifest-only cleanup cannot discover the orphan.

Use a durable per-identifier journal. Full reconciliation atomically records the union of desired and manifest identifiers before any Spotlight mutation. Each identifier stays pending until both the Spotlight mutation and manifest update succeed.

enum NoteIndexMutation: Codable, Sendable { case upsert, delete }

struct NoteIndexWork: Codable, Sendable {
    let identifier: String
    let revision: UInt64
    let mutation: NoteIndexMutation
}

enum NoteIndexResolution: Sendable {
    case upsert(NoteSnapshot)
    case delete
    case superseded
}

protocol NoteRepository: Sendable {
    // Each call writes the note/tombstone and latest per-ID outbox row in the
    // same database transaction, using a monotonically increasing revision.
    func commit(_ note: NoteSnapshot) async throws
    func commitDeletion(id: UUID) async throws
    func authorizedNote(
        id: UUID,
        protectionClass: FileProtectionType
    ) async throws -> NoteSnapshot?

    // These stage versioned work transactionally before Spotlight is touched.
    func stageIdentifiers(_ identifiers: [String]) async throws
    func stageFullReconciliation() async throws
    func pendingIndexWork() async throws -> [NoteIndexWork]
    func resolve(_ work: NoteIndexWork) async throws -> NoteIndexResolution
    func markIndexed(
        _ identifier: String,
        expectedRevision: UInt64
    ) async throws
    func markDeleted(
        _ identifier: String,
        expectedRevision: UInt64
    ) async throws
}

protocol IndexErrorReporting: Sendable {
    func record(_ error: any Error, operation: String) async
}

actor NoteSpotlightIndex {
    private let repository: any NoteRepository
    private let errors: any IndexErrorReporting
    private let index = CSSearchableIndex(
        name: "PrivateNotes",
        protectionClass: .complete
    )
    private var busy = false
    private var waiters: [CheckedContinuation<Void, Never>] = []

    init(
        repository: any NoteRepository,
        errors: any IndexErrorReporting
    ) {
        self.repository = repository
        self.errors = errors
    }

    func install(delegate: NoteIndexDelegate) {
        precondition(index.indexDelegate == nil)
        index.indexDelegate = delegate
    }

    func handlePartialReindexRequest(_ identifiers: [String]) async -> Bool {
        await enterGate()
        defer { leaveGate() }

        do {
            // If this commits, every requested ID has durable versioned work.
            try await repository.stageIdentifiers(identifiers)
        } catch {
            await errors.record(error, operation: "stage partial reindex")
            return false
        }

        do { try await drainJournal() }
        catch { await errors.record(error, operation: "drain partial reindex") }
        return true // Completed, or retry work remains durable.
    }

    func handleFullReindexRequest() async -> Bool {
        await enterGate()
        defer { leaveGate() }

        do {
            // One transaction scans source plus manifest and stages the union.
            try await repository.stageFullReconciliation()
        } catch {
            await errors.record(error, operation: "stage full reindex")
            return false
        }

        do { try await drainJournal() }
        catch { await errors.record(error, operation: "drain full reindex") }
        return true // Completed, or retry work remains durable.
    }

    private func drainJournal() async throws {
        for work in try await repository.pendingIndexWork() {
            switch try await repository.resolve(work) {
            case .upsert(let snapshot):
                try await index.indexSearchableItems([
                    NoteSearchItemFactory.makeItem(snapshot)
                ])
                try await repository.markIndexed(
                    work.identifier,
                    expectedRevision: work.revision
                )
            case .delete:
                try await index.deleteSearchableItems(
                    withIdentifiers: [work.identifier]
                )
                try await repository.markDeleted(
                    work.identifier,
                    expectedRevision: work.revision
                )
            case .superseded:
                continue
            }
        }
    }

    private func enterGate() async {
        guard busy else { busy = true; return }
        await withCheckedContinuation { waiters.append($0) }
    }

    private func leaveGate() {
        if waiters.isEmpty { busy = false }
        else { waiters.removeFirst().resume() }
    }
}

The app saves through repository.commit(_:) and deletes through commitDeletion(id:). Each transaction updates the source row or tombstone and the outbox together. There is no source-only commit window. On launch, the index owner drains pending work before accepting maintenance.

If indexing succeeds and markIndexed fails, work remains pending. Both completion methods delete the outbox row only when its revision still equals expectedRevision. A concurrent edit replaces it with a higher revision, so completion of the older operation cannot erase newer work. Delete tombstones use the same monotonic sequence. Full reconciliation stages versioned upserts for current notes and versioned tombstones for manifest-only IDs in one transaction, so it has equivalent recovery behavior.

3. Hydrate Through One Delegate

Some searchable values can match without being recoverable verbatim from the index. Implement CSSearchableIndexDelegate so Spotlight can request current, authorized records. Keep one actor responsible for the named custom index and retain one delegate.

import CoreSpotlight
import Foundation

final class OneShotAcknowledgement: @unchecked Sendable {
    private let lock = NSLock()
    private var callback: (() -> Void)?
    init(_ callback: @escaping () -> Void) { self.callback = callback }

    func call() {
        lock.lock()
        let callback = callback
        self.callback = nil
        lock.unlock()
        callback?()
    }
}

final class NoteIndexDelegate: NSObject,
    CSSearchableIndexDelegate,
    @unchecked Sendable {
    private let repository: any NoteRepository
    private let owner: NoteSpotlightIndex

    init(repository: any NoteRepository, owner: NoteSpotlightIndex) {
        self.repository = repository
        self.owner = owner
    }

    func searchableItems(
        forIdentifiers identifiers: [String],
        protectionClass: FileProtectionType
    ) async -> [CSSearchableItem] {
        guard protectionClass == .complete else { return [] }
        var items: [CSSearchableItem] = []
        for id in identifiers.compactMap(UUID.init(uuidString:)) {
            if let note = try? await repository.authorizedNote(
                id: id,
                protectionClass: protectionClass
            ) {
                items.append(NoteSearchItemFactory.makeItem(note))
            }
        }
        return items
    }

    func searchableIndex(
        _ index: CSSearchableIndex,
        reindexAllSearchableItemsWithAcknowledgementHandler done: @escaping () -> Void
    ) {
        let acknowledgement = OneShotAcknowledgement(done)
        Task { [owner, acknowledgement] in
            if await owner.handleFullReindexRequest() {
                acknowledgement.call()
            }
        }
    }

    func searchableIndex(
        _ index: CSSearchableIndex,
        reindexSearchableItemsWithIdentifiers identifiers: [String],
        acknowledgementHandler done: @escaping () -> Void
    ) {
        let acknowledgement = OneShotAcknowledgement(done)
        Task { [owner, acknowledgement] in
            if await owner.handlePartialReindexRequest(identifiers) {
                acknowledgement.call()
            }
        }
    }
}

struct NoteSearchComposition: Sendable {
    let owner: NoteSpotlightIndex
    let delegate: NoteIndexDelegate

    static func make(
        repository: any NoteRepository,
        errors: any IndexErrorReporting
    ) async -> NoteSearchComposition {
        let owner = NoteSpotlightIndex(
            repository: repository,
            errors: errors
        )
        let delegate = NoteIndexDelegate(
            repository: repository,
            owner: owner
        )
        await owner.install(delegate: delegate)
        return NoteSearchComposition(owner: owner, delegate: delegate)
    }
}

The composition root installs and retains the same delegate later passed to CoreSpotlightSource. The actor gate prevents reentrant index mutations while an async call is suspended. The callback’s index parameter stays outside the actor. The wrapper invokes the Objective-C acknowledgement once. A callback is acknowledged after reconciliation finishes or after the versioned outbox is durable. If staging fails, the actor records the error and returns false; the callback stays unacknowledged so Core Spotlight can request it again.

Hydration applies today’s authorization. If a note moved into a locked folder, return no item. An old index entry is never proof of access.

4. Configure SpotlightSearchTool

Apple’s sample creates a CoreSpotlightSource, selects attributes given to the model, and supplies the delegate. Focused guidance suits the on-device model’s smaller context window.

import CoreSpotlight
import FoundationModels

func makeNoteSearchTool(
    composition: NoteSearchComposition
) -> SpotlightSearchTool {
    var source = CoreSpotlightSource(
        searchableIndexDelegate: composition.delegate,
        fetchAttributes: [
            .title, .textContent, .keywords, .contentModificationDate
        ]
    )
    source.maximumResultCount = 20
    return SpotlightSearchTool(
        configuration: .init(
            sources: [.coreSpotlight(source)],
            guide: .focused(.documents)
        )
    )
}

Every fetched attribute consumes context and may expose unnecessary data. Begin with the smallest useful schema. Increase the limit or add fields only when an evaluation demonstrates a retrieval failure.

5. Stream Answers and Sources Separately

LanguageModelSession.streamResponse(to:) yields partial text. SpotlightSearchTool.searchResults yields actual search replies, possibly from several tool calls. Listen to both.

@MainActor
func answer(_ prompt: String, using tool: SpotlightSearchTool) async throws {
    sources = []
    response = ""

    let sourceTask = Task { @MainActor in
        var seen = Set<String>()
        for await reply in tool.searchResults {
            let items: [CSSearchableItem]
            switch reply.content {
            case .items(let values):
                items = values.map(\.item)
            case .scoredItems(let values):
                items = values.map(\.item.item)
            case .groupedItems(let groups):
                items = groups.values.flatMap { $0 }.map(\.item)
            case .count, .table, .statistic, .text:
                continue
            @unknown default:
                continue
            }
            for item in items where
                seen.insert(item.uniqueIdentifier).inserted {
                sources.append(item)
            }
        }
    }
    defer { sourceTask.cancel() }

    let session = LanguageModelSession(
        model: SystemLanguageModel.default,
        tools: [tool],
        instructions: """
        Answer only from notes returned by the search tool.
        Say plainly when the notes do not support an answer.
        Never invent a note, quotation, date, or decision.
        """
    )
    for try await chunk in session.streamResponse(to: prompt) {
        response = chunk.content
    }
}

The switch follows Apple’s sample for plain, scored, and grouped item replies and safely ignores aggregate or future cases. Deduplicate by uniqueIdentifier, and render source cards with the real title, modification date, and deep link. Never derive citations by parsing generated prose.

6. Handle Freshness, Privacy, and Fallback

RAG turns indexing failures into answer failures. Treat freshness as a product invariant:

  • Re-index after every committed edit and delete promptly.
  • Drain the journal on launch and after interrupted work.
  • Recheck authorization during hydration.
  • Show modification dates and let users open the source.
  • Remove account identifiers from Spotlight during sign-out.

If notes use SwiftData, index immutable snapshots rather than passing live models across actors. The same isolation rules apply when sharing SwiftData across apps and extensions.

Check SystemLanguageModel.default.availability whenever the feature starts. When generation is unavailable, offer ordinary Core Spotlight results without synthesis. Preserve cancellation when a second query replaces the first.

On-device generation can keep retrieved text local, but the selected LanguageModel determines processing. Disclose cloud routes accurately. Avoid logging raw prompts, bodies, or answers. Prefer availability reasons, result counts, latency buckets, hydration failures, and index generation.

7. Evaluate Retrieval and Generation Separately

Create versioned fixtures with prompts, available notes, and expected identifiers. Include paraphrases, dates, similar titles, deleted and unauthorized records, missing evidence, and prompt-injection attempts.

Measure four layers:

  1. Retrieval coverage: Did expected identifiers appear?
  2. Retrieval precision: How many returned items were irrelevant?
  3. Grounding: Does every material claim have source support?
  4. Answer behavior: Does the model admit missing evidence?

Also track time to first source, time to first text, cancellation, hydration, and stale-index recovery. Deterministic tests cover indexing, deletion, journal replay, authorization, deep links, and fallback. Evaluations cover model behavior. This follows a broader Swift Testing strategy for production apps: exact tests protect contracts while evaluations measure probabilistic quality.

Key Takeaways

  • Keep the database authoritative and Core Spotlight derived.
  • Journal every affected identifier before mutating the custom index.
  • Hydrate current authorized records through one retained delegate.
  • Give the model a small attribute set and bounded results.
  • Display returned items as sources beside streamed prose.
  • Provide ordinary search when generation is unavailable.
  • Evaluate retrieval, grounding, freshness, privacy, and latency separately.

Conclusion

SpotlightSearchTool removes much custom plumbing from private RAG on Apple platforms. It lets a Foundation Models session search an index your app owns, hydrate selected records, and expose returned items while an answer streams.

Trust still comes from app architecture. Stable identifiers, durable reconciliation, current authorization, small context budgets, honest fallback behavior, and repeatable evaluations make conversational search a dependable view over the user’s notes.

References