Back to writing
AI & LLMs·Nov 2025·7 min read

AI Baby Sleep Assistant, RAG and Palantir AIP

Built an AI-powered app to help parents navigate pediatric sleep challenges, grounded in academic research rather than generic advice. Combines semantic search, knowledge graphs, and LLMs via Palantir AIP to synthesise research papers into personalised guidance.

OK
Ofer Kulka
Staff Data & Analytics Engineer

Context

As parents, one of the biggest challenges we've faced is the lack of consistent, restful sleep, especially during the early years. Night wakings, sleep regressions, illness, and other unpredictable changes have often left us exhausted and frustrated.

We have two children: a baby nearing one year old and a four-year-old. Since the birth of our second, the struggle around sleep has intensified. Sudden shifts in sleep patterns often happen without any clear external trigger. We tried to make sense of it through online searches, advice from doctors, and our own intuition, but often ended up with more questions than answers.

So I turned to research. I began reviewing papers on child sleep development, looking for patterns, science-backed observations, and recommendations. That became the foundation for this project.

AI Baby Sleep Assistant, Workshop app interface
Fig 01, Workshop app interface

Tools and Methodologies

The app was developed using Palantir's Foundry and AIP Platform, leveraging several core services:

  • Pipeline Builder, data ingestion, transformation, LLM integration, chunking and embeddings
  • Vertex, Knowledge Graphs
  • Ontology Manager, semantic and kinetic modelling of data
  • Workshop, building the user-facing application
Palantir platform stack used: Pipeline Builder, Vertex, Ontology Manager, Workshop
Fig 02, Palantir Foundry + AIP stack

The Approach and Process

Data Collection

The project began with locating and selecting relevant research papers. I searched academic databases, health journals and health institutions for studies focused on baby, toddler and child sleep development. Papers were chosen based on three criteria:

  • Paper domain, baby/toddler/infant sleep
  • Number of citations (to ensure credibility)
  • Publication by recognised journals or health associations

Data Ingestion

I ingested a set of research papers (in PDF format) covering sleep development, regressions, sleep environments and contributing factors to healthy sleep.

Data Extraction

Using Pipeline Builder, I extracted text from PDFs first at the page level, then split each article into smaller structured chunks. Each chunk was tagged with metadata, mediaReference, timestamp, article ID, article name, page number and chunk number, to ensure downstream traceability and context preservation.

Data Processing

Chunking

Text was broken down article → pages → chunks for efficient processing while retaining semantic and contextual integrity (preserved via chunk metadata). Chunking is critical for the quality of LLM responses.

Embeddings

Each chunk was converted into a numerical vector representation preserving semantic context. At retrieval time the user's query is compared to embedded chunks; similar chunks are returned and compiled into a response.

LLM Integration

I designed a detailed prompt to guide the LLM's behaviour, what to consider, what to ignore, and how to compile a response. The goal: summarise each chunk while highlighting key sleep-related entities.

Entity Extraction & Deduplication

Sleep-related entities (e.g. "sleep regression," "sleep duration," "number of night wakings," "infant sleep") were extracted from the summarised text. Duplicates were removed to improve data quality.

Create Ontology Objects

I modelled the data into two Ontology Objects, Chunks and Entities, and defined the relationships between them. This enabled graph-based exploration and analysis.

Knowledge Graphs

Using Vertex and its built-in algorithms, I linked extracted Entities to their corresponding Chunks. The resulting Knowledge Graph lets users visually explore how different topics relate across the research dataset.

Knowledge Graph visualisation showing relationships between entities and chunks
Fig 03, Knowledge Graph: entities ↔ chunks

Function Creation

Built a function using AIP Logic to perform a semantic search over the chunks.

LLM Response

Use an LLM to provide the user with an appropriate response to their query.

The Application

The final step brought everything together into a single interactive app built with Palantir's Workshop service.

  • LLM Prompt Interface, users ask questions about child sleep
  • Chunk List, shows the relevant chunks used to generate the answer
  • Live Knowledge Graph, dynamically updates to show related chunks/entities
  • LLM Response, delivers a summarised, research-backed answer

Challenges and Solutions

One of the biggest hurdles was finding credible, free-to-access research. To address this, I prioritised papers with:

  • More than 20 citations
  • Publication by recognised journals or institutions

This ensured a solid, high-quality dataset to build from.

Results and Next Steps

This project helped deepen my understanding of Palantir's Platform, Pipeline Builder, the Ontology, LLM integration and prompt design, and semantic data processing and visualisation.

Most importantly, it showed me the power of a well-structured ontology: when the relationships, definitions, and properties are clearly defined, LLM outputs become more accurate and meaningful.

Limitations (so far)

The test responses from the app have been inconsistent, not yet meeting the quality I hoped for. What might need refinement:

  • The selection and diversity of research sources
  • Prompt design and structure
  • The ontology definitions themselves

Comparing answers with their referenced chunks, or reviewing them against paper abstracts, should reveal where improvements are needed.

Conclusion

This side project was born out of a deeply personal need: to make sense of the chaotic and often overwhelming experience of child sleep. By combining research, semantic modelling, and AI, I built an assistant that doesn't just regurgitate articles, it synthesises knowledge into practical, personalised advice.

There is still much to improve, but the foundation is in place and I'm excited to keep building and developing this further.

What's Next

  • Data Enrichment, ingest more research papers to expand the knowledge base.
  • Prompt Refinement, tweak prompts, adjust entities to improve responses.
  • Augment Ontology Objects, better descriptions of objects and properties to improve LLM understanding.
  • Use-case testing, run complex, real-world queries (e.g. "My child is 18 months old, has weaned off milk, and wakes up twice a night. What should I try?").
  • Compare LLMs, test with different LLMs beyond GPT-4.1 mini to assess output quality.
  • Expert Feedback, invite pediatric sleep researchers to test the app and provide insights on accuracy and usability.
Ofer Kulka

© 2026 Ofer Kulka · Data you can trust