Powering Investment Intelligence: A Deep Dive into Advanced Graph RAG with Neo4j, LLMs, and Vector Search

Powering Investment Intelligence: A Deep Dive into Advanced Graph RAG with Neo4j, LLMs, and Vector Search | Saumil Srivastava's Blog

1def create_investment_kg_constraints_updated(driver):
2    """Creates unique constraints on node properties."""
3    if not driver:
4        logging.error("Neo4j driver not available. Skipping constraint creation.")
5        return
6
7    try:
8        with driver.session(database="neo4j") as session:
9            constraints = [
10                "CREATE CONSTRAINT IF NOT EXISTS FOR (c:Company) REQUIRE c.ticker IS UNIQUE",
11                "CREATE CONSTRAINT IF NOT EXISTS FOR (e:ETF) REQUIRE e.ticker IS UNIQUE",
12                "CREATE CONSTRAINT IF NOT EXISTS FOR (n:NewsArticle) REQUIRE n.link IS UNIQUE",
13                "CREATE CONSTRAINT IF NOT EXISTS FOR (sec:Sector) REQUIRE sec.name IS UNIQUE",
14                "CREATE CONSTRAINT IF NOT EXISTS FOR (ind:Industry) REQUIRE ind.name IS UNIQUE",
15                "CREATE CONSTRAINT IF NOT EXISTS FOR (ac:AssetClass) REQUIRE ac.name IS UNIQUE",
16                "CREATE CONSTRAINT IF NOT EXISTS FOR (inv:Investor) REQUIRE inv.investorId IS UNIQUE",
17                "CREATE CONSTRAINT IF NOT EXISTS FOR (ent:__Entity__) REQUIRE ent.name IS UNIQUE"
18            ]
19            for constraint in constraints:
20                session.run(constraint)
21            logging.info("✅ Investment KG constraints created/ensured.")
22    except Exception as e:
23        logging.error(f"🛑 Error creating Investment KG constraints: {e}")
24

1// For Company
2MERGE (c:Company {ticker: $props.ticker})
3SET c = $props
4WITH c
5CALL apoc.create.addLabels(c, ['__Entity__']) YIELD node
6SET node.name = $props.ticker
7RETURN distinct 'company_done';
8
9// For Sector and relationship
10MERGE (s:Sector {name: $sector_name})
11ON CREATE SET s.name = $sector_name
12WITH s
13CALL apoc.create.addLabels(s, ['__Entity__']) YIELD node AS sector_node
14MATCH (c:Company {ticker: $company_ticker})
15MERGE (c)-[:OPERATES_IN_SECTOR]->(sector_node);
16

1def infer_news_sentiment(self, news_article_link: str, news_title: str) -> Optional[str]:
2    prompt_messages = [
3        {
4            "role": "system",
5            "content": "You are a financial news sentiment analyzer. Classify the sentiment of the given news headline as POSITIVE, NEGATIVE, or NEUTRAL. Return ONLY a JSON object with a single key 'sentiment' and one of these three values."
6        },
7        {
8            "role": "user",
9            "content": f"News Headline: \"{news_title}\"\n\nJSON:"
10        }
11    ]
12
13    llm_response = self._get_llm_response(prompt_messages)
14
15    if llm_response and "sentiment" in llm_response:
16        sentiment = llm_response["sentiment"].upper()
17        if sentiment not in ["POSITIVE", "NEGATIVE", "NEUTRAL"]:
18            sentiment = "NEUTRAL"
19
20        # Update Neo4j
21        with self.driver.session(database="neo4j") as session:
22            session.run("""
23                MATCH (n:NewsArticle {link: $link})
24                SET n.inferredSentiment = $sentiment
25            """, link=news_article_link, sentiment=sentiment)
26
27        logging.info(f"Updated sentiment for news '{news_title[:50]}...' to {sentiment}.")
28        return sentiment
29
30    return None
31

1def generate_company_investment_profile(self, company_ticker: str) -> Optional[str]:
2    # Fetch company data from Neo4j
3    # ... (query implementation)
4
5    prompt_messages = [
6        {
7            "role": "system",
8            "content": "You are a financial analyst providing concise investment profile summaries. Based on the provided company information and recent news sentiment, generate a 1-2 sentence 'investmentProfileSummary'. Focus on key characteristics, market position, or potential outlook. Output ONLY a JSON object with a single key 'investmentProfileSummary'."
9        },
10        {
11            "role": "user",
12            "content": f"""
13Company Name: {company_name}
14Sector: {sector}
15Industry: {industry}
16Existing Summary: {yfinance_summary[:1000]}
17Recent News Sentiment Trend: {overall_sentiment_hint}
18
19JSON:
20            """
21        }
22    ]
23
24    # Process and update Neo4j with the generated profile
25    # ... (implementation continues)
26

1from langchain_openai import OpenAIEmbeddings
2from langchain_community.vectorstores import Neo4jVector
3
4def create_neo4j_vector_index(driver, openai_api_key_for_lc, neo4j_uri, neo4j_username, neo4j_password):
5    embedding_node_property = 'embedding'
6    index_name = 'investmentEntityIndex'
7
8    # 1. Consolidate text into 'searchableDescription'
9    # Example for Company nodes:
10    # MATCH (c:Company)
11    # WHERE c.summary IS NOT NULL OR c.investmentProfileSummary IS NOT NULL
12    # SET c.searchableDescription = coalesce(c.name, "") + " | Industry: " +
13    #     coalesce(c.industry, "") + ". Summary: " +
14    #     coalesce(c.investmentProfileSummary, coalesce(c.summary, ""))
15
16    # 2. Create/Retrieve Vector Index
17    try:
18        if not os.getenv("OPENAI_API_KEY"):
19            os.environ["OPENAI_API_KEY"] = openai_api_key_for_lc
20
21        embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
22        vector_store = Neo4jVector.from_existing_graph(
23            embedding=embeddings,
24            url=neo4j_uri,
25            username=neo4j_username,
26            password=neo4j_password,
27            index_name=index_name,
28            node_label="__Entity__",
29            text_node_properties=["searchableDescription"],
30            embedding_node_property=embedding_node_property,
31            database="neo4j"
32        )
33        logging.info(f"✅ Vector index '{index_name}' created/retrieved successfully.")
34        return vector_store
35    except Exception as e:
36        logging.error(f"🛑 Failed to create/retrieve Neo4jVector index: {e}")
37        return None
38

1def generate_cypher_query(self, question: str) -> str:
2    system_prompt = f"""You are an expert Neo4j Cypher query writer...
3    Schema:
4    {self.schema}
5
6    Guidelines:
7    - Company nodes identified by 'ticker'
8    - ETF nodes identified by 'ticker'
9    - Relationships: OPERATES_IN_SECTOR, HOLDS_STOCK, INVESTED_IN (with 'stage' property)
10    - Use case-insensitive matching: toLower(node.property) CONTAINS toLower('search_term')
11    - For "latest news", ORDER BY n.publishTime DESC LIMIT 3
12    - Output ONLY the Cypher query
13    """
14
15    prompt_messages = [
16        {"role": "system", "content": system_prompt},
17        {"role": "user", "content": f"Natural language question: \"{question}\"\n\nCypher Query:"}
18    ]
19
20    # Generate Cypher using LLM
21    # ... (implementation)
22

1def retrieve_similar_nodes(self, query_text: str, k: int = 5, score_threshold: Optional[float] = None) -> List[Dict[str, Any]]:
2    logging.info(f"Performing vector similarity search for: '{query_text}' (k={k})")
3
4    # Uses self.vector_store.similarity_search_with_score
5    # Returns processed results with similarity scores
6    # ... (implementation)
7
8def synthesize_answer_from_vector_results(self, question: str, similar_nodes: List[Dict[str, Any]]) -> str:
9    prompt_messages = [
10        {
11            "role": "system",
12            "content": "You are a financial analyst assistant. Based on the user's question and the semantically similar entities retrieved from a knowledge graph, provide a concise and relevant natural language answer."
13        },
14        # ... (user prompt with question and context)
15    ]
16
17    # Generate answer using LLM
18    # ... (implementation)
19

Powering Investment Intelligence: A Deep Dive into Advanced Graph RAG with Neo4j, LLMs, and Vector Search

Table Of Contents

Introduction: Why Graph RAG for Investment Intelligence?

The Power of Graph RAG

Technical Challenges We'll Address

System Architecture: Building an Investment Intelligence Graph RAG

Building the Investment Knowledge Graph in Neo4j

Schema Definition

Entities (Nodes)

Relationships

Implementation: Constraints and Data Population

LLM-Powered Graph Enrichment

Sentiment Analysis Implementation

Investment Profile Generation

Vector Indexing for Semantic Search

Creating the Vector Index

Advanced Retrieval Strategies

Text2Cypher Retriever

Core Components:

Implementation Example:

Vector Similarity Retriever

Implementation:

The Router Retriever (Conceptual)

Performance, Scalability, and Best Practices

Performance Considerations

Common Implementation Pitfalls

Testing and Debugging Strategies

CI/CD and Versioning

Graph RAG vs. Vector RAG: When to Use Which?

Graph RAG Excels At:

Vector RAG Is Better For:

Infrastructure and Monitoring

Infrastructure Options

Neo4j Deployment

Resource Requirements

Technical Monitoring and Observability

Conclusion: Building ROI-Driven AI with Graph RAG

Key Technical Recommendations:

💻 Explore the Code & Go Deeper!

Additional Resources and References

Tools and Libraries

Documentation and Guides

Research Papers and Articles

Community Resources

Advanced Topics for Further Exploration

Subscribe to the Newsletter

Join the Newsletter

Table Of Contents