Safety Committee announces Paper Database Search Tool
You have probably heard me talk about it either at a Workshop or during our Podcast, but the moment has finally come. It's an early Christmas present from Santa to the flight test community. We are pleased to announce our roll out of our new LLM/AI Paper Database Search Tool. This would not be possible without the outstanding efforts of our AI sub-committee member Ryan Bowers.
How does this tool work? Here is a brief description that Ryan put together:
The tool uses a technique called Retrieval-Augmented Generation (RAG), which uses an LLM connected to a database of information. RAG is a technique to fine-tune an LLM to a specific database, in this case all the resources available on flighttestsafety.org, without the need for retraining, which would be expensive and infeasible for small-scale use.
When you ask a question, the RAG tool first generates a general answer and then backs it up with information retrieved directly from our database. The RAG does the following:
How does this tool work? Here is a brief description that Ryan put together:
The tool uses a technique called Retrieval-Augmented Generation (RAG), which uses an LLM connected to a database of information. RAG is a technique to fine-tune an LLM to a specific database, in this case all the resources available on flighttestsafety.org, without the need for retraining, which would be expensive and infeasible for small-scale use.
When you ask a question, the RAG tool first generates a general answer and then backs it up with information retrieved directly from our database. The RAG does the following:
- Database Creation: The files in the database are broken into text "chunks" which are converted into numerical vectors ("embeddings") that encode their semantic meaning.
- Document Retrieval: When you ask a query, it is converted into a numerical vector embedding in the same way as the database files. The retrieval system then searches through the embeddings of all corpus documents to find the document chunks that are most similar to the query.
- Context Assembly: The most relevant document chunks are retrieved and combined with your original question and conversation history to create a comprehensive context.
- Response Generation: A LLM (in our case, a lightweight variant of Gemini) uses the combined context from step 3 to generate a response to your query.
Because the context from step 3 only contains the most relevant content from the database, the LLM's response is tailored to your query and is less likely to be distracted by irrelevant content.
Our tool maintains conversation history. Each new message includes the previous conversation context, allowing for follow-up questions and coherent multi-turn discussions. When you are ready to change topics, you can clear the chat history. Like other LLM-based chat tools, this tool uses a system prompt which your query is appended to. This prompt shapes the model's behavior, tone, and things it is allowed and not allowed to say in response to your query.
We do have the ability to tune the system response through the prompt that is appended to the user’s question. Feedback is very much desired, and we can go in and tweak the prompt to try and improve results. Also, if you have additional resources you think we should add to the curated data, please let us know. It is a very simple process to add new data.
Access the Search Tool Here.
Enjoy and please give us feedback. Happy Holidays!
Our tool maintains conversation history. Each new message includes the previous conversation context, allowing for follow-up questions and coherent multi-turn discussions. When you are ready to change topics, you can clear the chat history. Like other LLM-based chat tools, this tool uses a system prompt which your query is appended to. This prompt shapes the model's behavior, tone, and things it is allowed and not allowed to say in response to your query.
We do have the ability to tune the system response through the prompt that is appended to the user’s question. Feedback is very much desired, and we can go in and tweak the prompt to try and improve results. Also, if you have additional resources you think we should add to the curated data, please let us know. It is a very simple process to add new data.
Access the Search Tool Here.
Enjoy and please give us feedback. Happy Holidays!
Stuart “Chia” Rogerson
Chairman, Flight Test Safety Committee