Improving Information Retrieval with fine-tuned Reranker


Colab Notebook on Github

Retrieval Augmented Generation (RAG) is often overhyped, leading to unmet expectations after implementation. While it may seem straightforward—combining a vector database with an LLM—achieving optimal performance is complex. RAG is easy to use but difficult to master, requiring deeper understanding and fine-tuning beyond basic setups.

More on RAG Agentic RAG with Redis, AWS Bedrock, and LlamaIndex

Advance RAG with fine-tuning

In the previous two blogs, we have covered how to fine-tune the initial retrieval part with BGE embedding model and Redis Vector Database.

Rerankers are specialized components in information retrieval systems that refine search results in a second evaluation stage. After an initial retrieval of relevant items, rerankers reorder the results to prioritize the most relevant ones, improving the quality and ranking of the final output

In this blogpost, we would focus on fine-tuning the reranker.

How Do Rerankers Improve RAG?

In a RAG system, a query is encoded into a vector and searched in a vector database containing document embeddings. The top-k matching documents are retrieved and used as context by a large language model (LLM) to generate a detailed, relevant response. This works well with small documents that fit within the LLM's context window. However, for large datasets, retrieved results may exceed the context window, causing information loss and reduced response quality.

To address this issue, you should employ a reranker to refine and prioritize the top-k matching documents before they are fed into the LLM.

The reranker reorders the retrieved documents based on relevance, ensuring the most pertinent information fits within the LLM's limited context window. This optimizes context usage, improving the accuracy and coherence of the response.


     Please find rest of the Blog in linkedin:


Comments

Post a Comment

Popular posts from this blog

Json to Avro

import org.apache.avro.Schema; import org.apache.avro.generic.GenericDatumReader; import org.apache.avro.generic.GenericDatumWriter; import org.apache.avro.generic.GenericRecord; import org.apache.avro.io.DatumReader; import org.apache.avro.io.Decoder; import org.apache.avro.io.DecoderFactory; import org.apache.avro.io.Encoder; import org.apache.avro.io.EncoderFactory; import java.io.ByteArrayOutputStream; import java.io.DataInputStream; import java.io.InputStream; import java.io.ByteArrayInputStream; public class JasonToAvro { /** * @param args * @throws Exception */ public static void main(String[] args) throws Exception { String json = "{\"username\":\"miguno\",\"tweet\":\"Rock: Nerf paper, scissors is fine.\",\"timestamp\": 1366150681 }"; String schemastr ="{ \"type\" : \"record\", \"name\" : \"twitter_schema\", \"namespace\" : \"com.miguno.avro\", \...
Read more

Parallel class hierarchies with Java Generic

Image
import java.util.ArrayList; import java.util.Collection; /* * Super class for Habitat hierarchy */ public abstract class Habitat <A extends Animal> { /* * A generic collection that can hold Animal * or any subclass of animal */ Collection<A> collection = new ArrayList<A>(); /* * add an Inhabitant to the collection. * should be overridden by subclass */ public abstract void addInhabitant( A animal); } /* * Aquarium class inherit the collection from * Habitat superclass. But limit the collection * to Fish type. */ public class Aquarium extends Habitat <Fish> { /* * (non-Javadoc) * @see Habitat#addInhabitant(Animal) */ @Override public void addInhabitant( Fish fish) { collection.add(fish); System.out.println(Aquarium.class); } } /* * Super class for Animal hierarchy */ public abstract class Animal { } public class Fish extends Animal { } public class Test { /** * @param args */ public static...
Read more

Hibernate CacheMode.IGNORE option

Recently, I have ran into a hibernate related issue in our production system. We have a nightly batch program that reads a lot of rows from one table, then does some conversion and writes to a file. It brings 5000 rows in chunk, still we saw the nightly program took the server down with outOfMemory error when processing large number of rows. After some investigation, I have found that the program brings out all the data in one session. Hibernate is holding all the objects in its session until session is closed. As a result, the GC can't clear any previous chunk of object from memory. Since the we are reading the data for readonly purpose, we set the CacheMode.IGNORE on the Query object. It prevents Hibernate to hold the reference of the objects in it's session. Here is snippet of the code. Query qry = session.createQuery(query).setCacheMode(CacheMode.IGNORE);
Read more