Praveen Singh

Google like search in Java : Hibernate Search

Introduction


HIBERNATE SEARCH: GOOGLE LIKE SEARCH IN JAVA

Search !
When i try to look back to my career as developer. Only module i can find out, common in all was search module.
We use search in every Enterprise Application, one way or another.
During the time, search module evolved beautifully, thanks to big search giant like Google, yahoo, ebay ...etc
Now, in software industry, whenever you build a search module, client always ask....can we have something like google ??
Providing a text box, where user can enter any serch key of any type is really the current and future of Search.
Though, ofeten it come on the price of high complexity, memory, low performance.
But the user experience or the ease of use it provide, is enough to make all disadvantage as nothing !

So why we should go for a serach module like Java and How to build in Java ?
My first video is going to start the process !




My second video going to cover details about the code.



Now, since we covered the basic part of Hibernate Search and before we move to advance part of it
It is best to enhance the user experience.
After All, Today's Software Industry is all about User Experience.

Step 1 :  Add "Did you mean ?" in search result.

My third video is going to cover all of them



Step 2 : Add in-time AJAX Search Suggestions

[coming soon !]

CODE





Before we start digging code, lets see how many jars we will be needed for this application
When i checked last time, i was using these jars



Lets have the code snippet one by one as it come in video

hibernate.cfg.xml


    <hibernate-configuration>
 <session-factory>
  
  <property name="connection.driver_class">com.mysql.jdbc.Driver</property>
  <property name="connection.url">jdbc:mysql://localhost:3306/hs</property>
  <property name="connection.username">root</property>
  <property name="connection.password">root</property>
  <property name="connection.pool_size">1</property>
  <property name="dialect">org.hibernate.dialect.MySQLDialect</property>
  <property name="current_session_context_class">thread</property>

  
  <property name="show_sql">true</property>

  
  <property name="hibernate.cache.use_second_level_cache">false</property>
  
  <property name="hibernate.search.default.directory_provider">org.hibernate.search.store.FSDirectoryProvider</property>
  <property name="hibernate.search.default.indexBase">c:/hsIndex</property>

  <mapping class="com.mylibrary.action.db.entitys.Book">
  <mapping class="com.mylibrary.action.db.entitys.Author">
 </mapping></mapping></session-factory>
</hibernate-configuration>



Book.java

package com.mylibrary.action.db.entitys;

import java.util.Date;
import java.util.HashSet;
import java.util.Set;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.FetchType;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;
import javax.persistence.ManyToMany;

import org.hibernate.search.annotations.DateBridge;
import org.hibernate.search.annotations.DocumentId;
import org.hibernate.search.annotations.Field;
import org.hibernate.search.annotations.Index;
import org.hibernate.search.annotations.Indexed;
import org.hibernate.search.annotations.IndexedEmbedded;
import org.hibernate.search.annotations.NumericField;
import org.hibernate.search.annotations.Resolution;
import org.hibernate.search.annotations.Store;



@Entity
@Indexed
public class Book {

 @Id
 @GeneratedValue
 @DocumentId
 private Integer id;

 @Field(index = Index.TOKENIZED, store = Store.NO)
 private String title;

 @Field(index = Index.UN_TOKENIZED, store = Store.NO)
 private String isbn;

 @Field(index = Index.TOKENIZED, store = Store.NO)
 private String publisher;

 @Field(index = Index.TOKENIZED, store = Store.NO)
 @Column(name = "description", length = 1500)
 private String desc;

 @Field
 @NumericField(precisionStep = 6)
 // precisionStep : default value = 4
 private float price;

 @Field(name = "dateOfPublication", index = Index.UN_TOKENIZED)
 @DateBridge(resolution = Resolution.YEAR)
 private Date dateOfPublication;

 @ManyToMany(cascade = { javax.persistence.CascadeType.ALL }, fetch = FetchType.EAGER)
 @IndexedEmbedded
 private Set authors = new HashSet();

 public Integer getId() {
  return this.id;
 }

 public void setId(Integer id) {
  this.id = id;
 }

 public String getTitle() {
  return this.title;
 }

 public void setTitle(String title) {
  this.title = title;
 }

 public Set getAuthors() {
  return this.authors;
 }

 public void setAuthors(Set authors) {
  this.authors = authors;
 }

 public String getDesc() {
  return this.desc;
 }

 public void setDesc(String desc) {
  this.desc = desc;
 }

 public float getPrice() {
  return price;
 }

 public void setPrice(float price) {
  this.price = price;
 }

 public Date getDateOfPublication() {
  return dateOfPublication;
 }

 public void setDateOfPublication(Date dateOfPublication) {
  this.dateOfPublication = dateOfPublication;
 }

 public String getIsbn() {
  return isbn;
 }

 public void setIsbn(String isbn) {
  this.isbn = isbn;
 }

 public String getPublisher() {
  return publisher;
 }

 public void setPublisher(String publisher) {
  this.publisher = publisher;
 }

 @Override
 public String toString() {
  return "Book [id=" + id + ", title=" + title + ", isbn=" + isbn
    + ", publisher=" + publisher + ", desc=" + desc + ", price="
    + price + ", dateOfPublication=" + dateOfPublication
    + ", authors=" + authors + "]";
 }

 @Override
 public int hashCode() {
  final int prime = 31;
  int result = 1;
  result = prime * result + ((id == null) ? 0 : id.hashCode());
  return result;
 }

 @Override
 public boolean equals(Object obj) {
  if (this == obj)
   return true;
  if (obj == null)
   return false;
  if (getClass() != obj.getClass())
   return false;
  Book other = (Book) obj;
  if (id == null) {
   if (other.id != null)
    return false;
  } else if (!id.equals(other.id))
   return false;
  return true;
 }

}



Author.java

package com.mylibrary.action.db.entitys;

import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;
import org.hibernate.search.annotations.Field;
import org.hibernate.search.annotations.Index;
import org.hibernate.search.annotations.Indexed;
import org.hibernate.search.annotations.Store;

@Entity
@Indexed
public class Author
{

  @Id
  @GeneratedValue
  private Integer id;

  @Field(index=Index.TOKENIZED, store=Store.NO)
  private String name;

  public Integer getId()
  {
    return this.id;
  }

  public void setId(Integer id)
  {
    this.id = id;
  }

  public String getName()
  {
    return this.name;
  }

  public void setName(String name)
  {
    this.name = name;
  }

  public String toString()
  {
    return this.name;
  }
}



SearchController.java

package com.mylibrary.controller;

import java.io.IOException;
import java.util.Set;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.apache.lucene.queryParser.ParseException;

import com.mylibrary.action.db.dao.SearchDAO;
import com.mylibrary.action.services.SearchSugestionMgr;

@SuppressWarnings("serial")
public class SearchController extends HttpServlet {

  @Override
  public void init() throws ServletException {
    //SearchSugestionMgr.getInstance();
  }

  @SuppressWarnings("rawtypes")
  protected void doGet(HttpServletRequest request,
      HttpServletResponse response) throws ServletException, IOException {
    String searchKey = request.getParameter("searchKey");
    Set result = null;
    try {
      SearchDAO searchDAO = new SearchDAO();
      result = searchDAO.doSearching(searchKey);
      System.out.println("size of result : " + result);
    } catch (ParseException e) {
      e.printStackTrace();
    }
    request.setAttribute("result", result);
    request.setAttribute("searchKey", searchKey);

    //this.addSearchSuggestion(request, searchKey);
    request.getRequestDispatcher("Results.jsp").forward(request, response);
  }

  private void addSearchSuggestion(HttpServletRequest request,
      String searchKey) {
    String[] suggestions = null;
    try {
      suggestions = SearchSugestionMgr.getInstance().serachSuggetion(
          searchKey);
    } catch (Exception e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
    }

    request.setAttribute("serachSuggestion", suggestions);
  }
}



SearchDAO.java

package com.mylibrary.action.db.dao;

import java.util.Calendar;
import java.util.Date;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

import org.apache.lucene.queryParser.ParseException;
import org.hibernate.Session;
import org.hibernate.search.FullTextSession;
import org.hibernate.search.Search;
import org.hibernate.search.query.dsl.QueryBuilder;

import com.mylibrary.action.db.entitys.Book;
import com.mylibrary.action.db.manger.SessionManger;

public class SearchDAO {

  private Date getDate(String year) {
    try {
      Calendar calendar = Calendar.getInstance();
      calendar.set(Calendar.YEAR, Integer.parseInt(year));
      return calendar.getTime();
    } catch (NumberFormatException nfe) {
      return null;
    }
  }

  @SuppressWarnings({ "rawtypes", "unchecked" })
  public Set doSearching(String searchKey) throws ParseException {
    System.out.println("searchKey : " + searchKey);
    Session session = SessionManger.openSession();
    FullTextSession fullTextSession = Search.getFullTextSession(session);
    final QueryBuilder b = fullTextSession.getSearchFactory()
        .buildQueryBuilder().forEntity(Book.class).get();

    org.apache.lucene.search.Query luceneQuery = b.keyword()
        .onFields("title", "publisher", "desc", "isbn", "authors.name")
        .matching(searchKey).createQuery();
    org.hibernate.Query fullTextQuery = fullTextSession
        .createFullTextQuery(luceneQuery);
    List result = fullTextQuery.list();
    
    
    Date date = this.getDate(searchKey);
    List resultByDate = null;
      
    if (date != null) {
      luceneQuery = b.keyword().onField("dateOfPublication")
          .matching(this.getDate(searchKey)).createQuery();
      fullTextQuery = fullTextSession.createFullTextQuery(luceneQuery);
       resultByDate = fullTextQuery.list();
    }
    // Because od some reason, we are getting duplicate result.
    // This fix is just to get unique result.
    Set uniqueResult = new HashSet();
    if (!(result == null || result.size() == 0)) {
      uniqueResult.addAll(result);
    }
    if (!(resultByDate == null || resultByDate.size() == 0)) {
      uniqueResult.addAll(resultByDate);
    }
    return uniqueResult;
  }
  
}




SessionManger.java

package com.mylibrary.action.db.manger;

import org.hibernate.Session;
import org.hibernate.SessionFactory;
import org.hibernate.cfg.Configuration;

public class SessionManger {

  private static final SessionFactory sessionFactory = buildSessionFactory();

  private static SessionFactory buildSessionFactory() {
    try {
      // Create the SessionFactory from hibernate.cfg.xml
      return new Configuration().configure().buildSessionFactory();
    } catch (Throwable ex) {
      // Make sure you log the exception, as it might be swallowed
      System.err.println("Initial SessionFactory creation failed." + ex);
      throw new ExceptionInInitializerError(ex);
    }
  }

  public static SessionFactory getSessionFactory() {
    return SessionManger.sessionFactory;
  }

  public static Session getCurrentSession() {
    return SessionManger.sessionFactory.getCurrentSession();
  }

  public static Session openSession() {
    return SessionManger.sessionFactory.openSession();
  }

}






Test.html


    <title>IcodingClubSearch</title>


<img alt="icodingclub_banner" src="ICCS.png" style="margin-left: 100px;" />
<form action="SearchController"><input name="searchKey" size="50" style="margin-left: 100px; margin-top: 5px;" type="text" />

<input style="margin-left: 200px; margin-top: 5px;" type="submit" value="Search" />

</form>





Results.jsp

<%@page import="java.util.List"%>
<%@page import="java.util.Calendar"%>
<%@page import="com.mylibrary.action.db.entitys.Author"%>
<%@page import="com.mylibrary.action.db.entitys.Book"%>
<%@page import="java.util.Set"%>
<%@ page language="java" contentType="text/html; charset=ISO-8859-1"
  pageEncoding="ISO-8859-1"%>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>

<style type="text/css">
.resultTable {
  border: 1px gray solid;
  width: 700px;
  border-spacing: 0px;
  border-collapse: collapse;
  margin-bottom: 50px;
  margin-left: 100px;
}

.oddRow {
  border-right: 1px gray solid;
  background-color: #F3F3F3;
}
</style>

<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>IcodingClubSearch</title>
</head>
<body>
<%
  String searchKey = (String) request.getAttribute("searchKey");
%>
<img style="margin-left: 100px" alt="icodingclub_banner" src="ICCS.png">
<form action="SearchController"><input
  style="margin-left: 100px; margin-top: 5px" name="searchKey"
  type="text" size="50" value="<%=searchKey%>"> <br>
<input style="margin-left: 200px; margin-top: 5px" type="submit"
  value="Search"> <br>

<%
  String[] suggestions = (String[]) request
  .getAttribute("suggestions");

  if (!(suggestions == null || suggestions.length == 0)) {
out.print("<label style='margin-left: 100px;margin-top: 5px;margin-bottom: 5px;color: maroon;'>Did you mean ? : </label>");
for (String suggestion : suggestions) {
  out.print("<a href='DidYouMeanSearch?searchKey="
  + suggestion + "'>" + suggestion + "</a>, ");
}
  }
%> <br>

<h2 style="color: maroon; margin-left: 100px">Search Results</h2>
<%
  Set books = (Set) request.getAttribute("result");
  for (Object obj : books) {
Book book = (Book) obj;
out.print("<table class='resultTable'>");
out.print("<tr>");
out.print("<td>Title</td>");
out.print("<td>");
if (book.getTitle().toLowerCase()
.indexOf(searchKey.toLowerCase()) > -1) {
  out.print(book.getTitle().replaceAll("(?i)" + searchKey,
  "<b>" + searchKey + "</b>"));
} else {
  out.print(book.getTitle());
}

out.print("</td>");
out.print("</tr>");
out.print("<tr class='oddRow'>");
out.print("<td>Author</td>");
out.print("<td>");
for (Author author : book.getAuthors()) {
  if (author.getName().toLowerCase()
  .indexOf(searchKey.toLowerCase()) > -1) {
out.print(author.getName().replaceAll(
"(?i)" + searchKey, "<b>" + searchKey + "</b>"));
  } else {
out.print(author.getName());
  }
}

out.print("</td>");
out.print("</tr>");

out.print("<tr>");
out.print("<td>ISBN</td>");
out.print("<td>");
if (book.getIsbn().toLowerCase()
.indexOf(searchKey.toLowerCase()) > -1) {
  out.print(book.getIsbn().replaceAll("(?i)" + searchKey,
  "<b>" + searchKey + "</b>"));
} else {
  out.print(book.getIsbn());
}
out.print("</td>");
out.print("</tr>");

out.print("<tr class='oddRow'>");
out.print("<td >Price</td>");
out.print("<td >");
String price = "" + book.getPrice();
if (price.toLowerCase().indexOf(searchKey.toLowerCase()) > -1) {
  out.print(price.replaceAll("(?i)" + searchKey, "<b>"
  + searchKey + "</b>"));
} else {
  out.print(price);
}
out.print("</td>");
out.print("</tr>");

out.print("<tr>");
out.print("<td >Publisher</td>");
out.print("<td >");
if (book.getPublisher().toLowerCase()
.indexOf(searchKey.toLowerCase()) > -1) {
  out.print(book.getPublisher().replaceAll(
  "(?i)" + searchKey, "<b>" + searchKey + "</b>"));
} else {
  out.print(book.getPublisher());
}
out.print("</td>");
out.print("</tr>");

out.print("<tr class='oddRow'>");
out.print("<td >Year Of Publication</td>");
out.print("<td >");
Calendar cal = Calendar.getInstance();
cal.setTime(book.getDateOfPublication());
String year = "" + cal.get(Calendar.YEAR);
if (year.toLowerCase().indexOf(searchKey.toLowerCase()) > -1) {
  out.print(year.replaceAll("(?i)" + searchKey, "<b>"
  + searchKey + "</b>"));
} else {
  out.print(year);
}
out.print("</td>");
out.print("</tr>");

out.print("<tr>");
out.print("<td >Discription</td>");
out.print("<td >");
if (book.getDesc().toLowerCase()
.indexOf(searchKey.toLowerCase()) > -1) {
  out.print(book.getDesc().replaceAll("(?i)" + searchKey,
  "<b>" + searchKey + "</b>"));
} else {
  out.print(book.getDesc());
}
out.print("</td>");
out.print("</tr>");
out.print("</table>");
out.print("</br>");
out.print("</br>");
  }
%>

</body>
</html>




SearchSugestionMgr.java

package com.mylibrary.action.services;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.ArrayList;
import java.util.List;

import org.apache.lucene.search.spell.PlainTextDictionary;
import org.apache.lucene.search.spell.SpellChecker;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.FSDirectory;

public class SearchSugestionMgr {

  private static int SUGGESTION_NO = 3;

  private SpellChecker spellChecker;

  private static SearchSugestionMgr sugestionMgr = new SearchSugestionMgr();

  private SearchSugestionMgr() {
    try {

      File dir = new File(SearchSugestionMgr.class.getResource(
          "/data/indexFiles/").toURI());

      Directory directory = FSDirectory.open(dir);

      this.spellChecker = new SpellChecker(directory);
      this.spellChecker.indexDictionary(new PlainTextDictionary(new File(
          SearchSugestionMgr.class.getResource(
              "/data/dictionary/dictionary.txt").toURI())));
    } catch (Exception e) {
      e.printStackTrace();
    }
  }

  public static SearchSugestionMgr getInstance() {
    return SearchSugestionMgr.sugestionMgr;
  }

  public String[] serachSuggetion(String searchKey) throws Exception {

    String[] suggestions = this.spellChecker.suggestSimilar(searchKey,
        SearchSugestionMgr.SUGGESTION_NO);
    return suggestions;
  }

  private String getAllSuggestions() throws URISyntaxException {
    // String wordSuggestion =
    // this.fileToString(SearchSugestionMgr.class.getResource("/data/dictionary/dictionary.txt").toURI());
    String wordSuggestion = "";
    List serachSuggestionFromDB = this.getSuggestionFromDB();

    for (String string : serachSuggestionFromDB) {
      wordSuggestion += string + "\\n";
    }

    return wordSuggestion;
  }

  private String fileToString(URI filePath) {
    String fileData = "";
    String fileDataTemp = "";
    BufferedReader reader = null;

    try {
      reader = new BufferedReader(new FileReader(new File(filePath)));
    } catch (FileNotFoundException e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
    }
    try {
      while ((fileDataTemp = reader.readLine()) != null) {
        fileData += fileDataTemp;
      }
    } catch (IOException e) {
      e.printStackTrace();
    }
    return fileData;
  }

  private List getSuggestionFromDB() {
    List serachSuggestionFromDB = new ArrayList();

    serachSuggestionFromDB.add("Chetan");
    serachSuggestionFromDB.add("Dan Brown");

    return serachSuggestionFromDB;
  }

}


If you love eclipse, i have uploaded the application in form of eclipse project.
You can download it from this DOWNLOAD link

[MORE ! ......COMING SOON]

8 comments:

  1. Can you convert that file to maven project?
    Sorry Im still newbie...

    ReplyDelete
  2. Hi Toko,
    Making a maven project/file is bit long process.

    I have been in industry from long time and can guarantee you that this zipped eclipse project is far better than that.
    If you are not sure how to upload in eclipse, i can guide you with that or you can do a simple google search.

    Just let me know, i will glad to help you

    ReplyDelete
    Replies
    1. Hi, I unzip that file in STS ( Eclipse) but it give me error. Like this:

      HTTP Status 404 - /SearchController

      --------------------------------------------------------------------------------

      type Status report

      message /SearchController

      description The requested resource (/SearchController) is not available.


      --------------------------------------------------------------------------------

      Apache Tomcat/7.0.12

      Delete
  3. My name is Zacky, Toko means shop in Indonesian language :) (its just my undone blog,he2)

    Ok, I'm sure that this zipped project is better.
    But I need a sample to make google-like search that take data from database.
    I work in industry too, IT Consultant, but I just a fresh-graduated who new to hibernate.
    Now I have a task to make google-like search that build in Maven.

    Could you help me please?
    I'm still blind...

    ReplyDelete
  4. I'm going to steal your code. I mean i'm going to use this in my project. Thanks. I been googleing about hibernate search for two days and not successful in implementing it. I think i need time to understand this topic. I have a project due next week and i need this functionality really bad. thanks again!

    ReplyDelete
  5. Hi Praveen,

    How do you compile this. I'm using STS. Thanks!

    ReplyDelete
  6. Enjoyed your approach to explaining how it works, hope to see more blog posts from you. thank you!

    Hibernate Online Training | Java Online Training


    Hibernate Training in Chennai Java Training in Chennai

    ReplyDelete