Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
772740e
starting to implement LLM API call into search function
Nov 4, 2024
a66bd3d
Updated README
Nov 4, 2024
cb69682
LLM Querying and chunking working on backend
Nov 6, 2024
2da2bbc
returning docs that match LLM
Nov 6, 2024
80fbb9c
Working draft of front end implementation
Nov 7, 2024
8f9ba63
front end updates
Nov 8, 2024
fe2ffa4
AI search feature implemented
Nov 8, 2024
4de1926
AI search feature is fully tested and functioning
Nov 8, 2024
4bbfc02
updated packages and working node version
Nov 8, 2024
d88cf52
For deployment
Nov 8, 2024
f04af3a
for vercel deployment
Nov 8, 2024
40e564b
For vercel deployment
Nov 8, 2024
baffc5e
Vercel deployment
Nov 8, 2024
633d261
vercel deployment
Nov 8, 2024
2c16ed3
vercel deployment
Nov 8, 2024
2417e13
vercel deployment
Nov 8, 2024
0f24709
Vercel deployment
Nov 8, 2024
2dfe5b0
Trying to get vercel to work
Nov 8, 2024
cc026d7
Add logic to parse base64 auth string for Vercel
Nov 8, 2024
fdf5480
Add logic to parse base64 auth string for Vercel
Nov 8, 2024
790b008
bug fix for vercel
Nov 8, 2024
5154374
vercel test
Nov 8, 2024
686f157
reverting for vercel
Nov 8, 2024
1709486
vercel testing
Nov 8, 2024
f8de6af
reverting vercel test
Nov 8, 2024
ef6ffc3
fixes for vercel deployment
Nov 9, 2024
b65c564
trying to figure out why vercel isn't deploying
Nov 9, 2024
089142b
quick fix
Nov 9, 2024
6bd5bcf
bug testing
Nov 9, 2024
7a1dfec
changing fallback function since it's not working on vercel
Nov 9, 2024
3d88641
removing requirewithfallback since it breaks vercel
Nov 9, 2024
08a3ecf
forcing https for google auth
Nov 9, 2024
c9787e1
forcing https for google aguth
Nov 9, 2024
de4ba49
Small bug fix
Nov 9, 2024
4ed09e9
disable user auth
Nov 9, 2024
125783f
bug fixes and removing console log
Nov 9, 2024
5265df3
needed for deployment
Nov 9, 2024
2f276ce
mobile friendly styling
Nov 9, 2024
7835021
mobile search looks good
Nov 9, 2024
f867aa1
bug fix
Nov 9, 2024
5780f41
GIF for README
Nov 9, 2024
e96b53a
Demo for README
Nov 9, 2024
a534cbb
Final documentation
Nov 9, 2024
098ace5
Final documentation
Nov 9, 2024
4d4c31c
Final documentation
Nov 9, 2024
8adc0b7
AI search feature demo gif
Nov 9, 2024
3fad396
resizing demo
Nov 9, 2024
3ff8c28
Updated README
Nov 9, 2024
6cfa7cf
Updated README
Nov 9, 2024
f53af82
Updated README
Nov 9, 2024
7555145
Updated null answer message
Nov 9, 2024
9a6b8f5
linting for style
Nov 9, 2024
cddba5f
removing placeholder
Nov 9, 2024
4ae6c9e
returning object instead of list for consistency
Nov 9, 2024
34e1f01
added ai search test
Nov 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added AISearchDemo.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,21 @@
# Introducing AI powered search for Library!

Users can now search the library with AI by simply asking a question in the search box!

You can find a full demo [here](https://llm-docs-library.vercel.app/).
<br></br>
**Features:**
- Returns a concise summary along with relevant documents, creating a more powerful search than keywords.
- Fetches entire document base for AI query using existing infrastructure that supports caching.
- Supports chunking of AI requests for models with smaller token limits.
- Can query multiple times and combine responses for greater accuracy using the `LLM_API_CALLS` environment variable.
- Makes API and fetch requests in parallel.
- Defaults to a null answer to avoid hallucinations.
- Uses both desktop and mobile-friendly front-end animations.

<img src="https://github.com/nshaff3r/llm-docs-library/blob/main/AISearchDemo.gif" alt="AI search feature demo gif" width="558.75px" max-height="600px">


Library ![Supported node versions](https://img.shields.io/badge/dynamic/json?color=informational&label=node&query=%24.engines.node&url=https%3A%2F%2Fraw.githubusercontent.com%2Fnytimes%2Flibrary%2Fmain%2Fpackage.json) [![Tests](https://github.com/nytimes/library/actions/workflows/test.yaml/badge.svg)](https://github.com/nytimes/library/actions/workflows/test.yaml)
========

Expand Down
13 changes: 13 additions & 0 deletions index.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
indexes:
- kind: LibraryViewTeam
properties:
- name: userId
direction: asc
- name: viewCount
direction: desc
- kind: LibraryViewTeam
properties:
- name: userId
direction: asc
- name: lastViewedAt
direction: desc
2 changes: 2 additions & 0 deletions layouts/categories/default.ejs
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
<!DOCTYPE html>
<html>
<%- include('partials/head', {formatUrl, pathPrefix}) %>
<script src="assets/scripts/animations.js"></script>
<link rel="stylesheet" href="assets/css/customstyles.css">
<body>
<h1 class="visually-hidden">Default template</h1>
<%- include('partials/header', {formatUrl}) %>
Expand Down
1 change: 0 additions & 1 deletion layouts/pages/index.ejs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
<html>
<%- include('partials/head', {title: template('branding.prettyName'), formatUrl, pathPrefix}) %>
<body>

<header id="masthead" class="masthead" role="banner">
<div class="container">
<%- include('partials/userTools') %>
Expand Down
45 changes: 45 additions & 0 deletions layouts/pages/llmquery.ejs
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
<!DOCTYPE html>
<html>
<head>
<%- include('partials/head', {title: template('search.results.title', locals.q), formatUrl, pathPrefix}) %>
</head>
<body>
<%- include('partials/header', {parentLinks: [], title: template('search.results.title', locals.q)}) %>
<%- include('partials/nav', {formatUrl}) %>
<div class="g-body">
<div class="g-main-content" id="g-search-page">
<h3 class="search-kicker">
Showing <%- '<span class="animated-gradient-text">AI</span>' %> search results for '<%= locals.q %>':
</h3>
<div id="llm-response-box">
<div id="llm-response">
<%= locals.llmResponse %>
</div>
<% if (locals.results && results.length) { %>
<div id="disclaimer">
AI can be incorrect. Always check this response against the documents and the non-AI search.
</div>
<% } %>
</div>
<% if (locals.results && results.length) { %>
<% results.forEach((res) => { %>
<div class="search-result">
<h3><a href="<%= formatUrl(res.path) %>"><%= res.prettyName %></a></h3>
<p>
<% if (res.folder) { %>
<%- template('search.results.label.folder', formatUrl(res.folder.path), res.folder.prettyName || 'Home')%>&nbsp;
<% } %>
<% if (res.lastModifyingUser && res.lastModifyingUser.displayName) { %>
<%- template('search.results.label.person', res.lastModifyingUser.displayName) %>
<% } %>
<%= res.lastUpdated %>
</p>
</div>
<% }) %>
<% } %>
</div>

<%- include('partials/footer', { pageType: 'search', formatUrl }) %>
</div>
</body>
</html>
2 changes: 2 additions & 0 deletions layouts/partials/head.ejs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
<![endif]-->

<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
<script src="/assets/scripts/animations.js"></script>
<link rel="stylesheet" href="/assets/css/customstyles.css">

<link rel="stylesheet"
href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/10.0.0/styles/default.min.css">
Expand Down
6 changes: 3 additions & 3 deletions layouts/partials/search.ejs
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
<div class="search-topper">
<form method="GET" action="<%= formatUrl('/search') %>" class="search-container">
<% const placeholder = template('search.placeholder')%>
<% const placeholder = 'Search... (or ask a question with AI)'%>
<% const focus = locals.focus || '' %>
<% const msgOnFocus = placeholder || '' %>
<input <%= focus %> id="search-box" class="twitter-typeahead" type="text" name="q" value="<%= locals.q %>" placeholder="<%- placeholder %>" />
<button class="icon"><i class="fa fa-search"></i></button>
<button class="icon" id="searching-button"><i id="searching-icon" class="fa fa-search"></i></button>
</form>

<% const style = locals.style || 'plaintext' %>
Expand All @@ -13,4 +13,4 @@
<%# <a href="#"><button type="button" class="btn-plaintext">Onboarding Guide</button></a>%>
</div>

</div>
</div>
Loading