For creating the index from the existing data, we need to create the index. Isn’t it a better idea to index each data when its created, to index the real time data we need to open the index which was created earlier, rest of the other things remains same as discussed in the Zend lucene Search – part1.
<?php
require_once 'Zend/Search/Lucene.php';
$index = Zend_Search_Lucene::open("/var/www/lucene-data/blog-index");
$doc = new Zend_Search_Lucene_Document();
$doc->addField(Zend_Search_Lucene_Field::Keyword('url', "http://ganeshhs.com/url-3"));
$doc->addField(Zend_Search_Lucene_Field::UnIndexed('articleId', 3));
$doc->addField(Zend_Search_Lucene_Field::UnIndexed('postedDateTime', "20007-12-29 01:40:00"));
$doc->addField(Zend_Search_Lucene_Field::Text('title', "Porting PHP to Javascript : php js"));
$doc->addField(Zend_Search_Lucene_Field::UnStored('contents',
"During graduation got interested in web technology, to kick start i started reading html, javascript."));
$doc->addField(Zend_Search_Lucene_Field::Text('category', "Javascript"));
$index->addDocument($doc);
$index->commit();
$index->optimize();
6 Responses
Neeraj Kumar
December 13th, 2011 at 8:31 am
1HI Ganesh,
I am facing an issue while using Zend Search Lucene as described below:-
I tried creating zend lucene document for users contained in my database table having more than 65000 records. I added the below method in Bootstrap.php
protected function _initZendLucene()
{
$userObj = new ZF_User();
$data = $userObj->fetchAll();
foreach ($data as $d){
$index = Zend_Search_Lucene::create(APPLICATION_PATH . ‘/../public/search’);
$doc = new Zend_Search_Lucene_Document();
$doc->addField(
Zend_Search_Lucene_Field::text(
‘first_name’, $d->getFirstName()) );
$index->addDocument($doc);
$doc = new Zend_Search_Lucene_Document();
$doc->addField(
Zend_Search_Lucene_Field::text(
‘last_name’, $d->getLastName()) );
$index->addDocument($doc);
}
$index->commit();
}
// ZF_User() is my model for user table
At indexController.php i just tried calling it.
public function indexAction()
{
$index = Zend_Search_Lucene:pen(APPLICATION_PATH . ‘/../public/search’);
$results = $index->find(‘first_name:neeraj’);
foreach ( $results as $result ) {
echo “name :”.$result->first_name.”\n”;
}
}
But the issue is it takes so much long time that when i try to browse the home page then script execution stopped. I also increased max_execution_time as well as memory_limit in the php.ini file but it didn’t work for me.
Isn’t there any solution to use lucene or any other searching mechanism for a bulky data record except Full Text search as my table engine is INNODB.
ganeshhs
December 14th, 2011 at 3:30 am
2Hey Niraj,
You should index the data using crontab script and later use real time indexing when you are trying to add or delete or update.
- Ganesh
Neeraj Kumar
December 14th, 2011 at 7:06 am
3Ganesh,
I put the index code in cron. When i tried fetching first 500 users from database then it worked fine but there are 65000+ records in my database use table and when i tried indexing for all records then got the error
PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to all
ocate 35 bytes) in C:\wamp\www\zendproject\application\models\ZF\User.php on lin
e 250
Where as i have configured memory_limit 512M in php.ini
Can you please suggest on this.
- Neeraj
ganeshhs
December 17th, 2011 at 8:01 am
4Neeraj,
Try optimizing it the index, read the section Index optimization http://framework.zend.com/manual/en/zend.search.lucene.index-creation.html
Try seeing disk I/O and RAM and CPU cycles when you run that. I doubt the resources what is allocated is less.
ganeshhs
December 17th, 2011 at 8:03 am
5I have preferred to move to apache solr if the data size is more, i am getting that platform developed in house for my products and client projects
Neeraj Kumar
December 19th, 2011 at 5:24 am
6Okay, Can you please provide example of implementing solr in zend framework?
RSS feed for comments on this post · TrackBack URI
Leave a reply
Categories
Archives
Blogroll
Ganesh H S is proudly powered by WordPress - BloggingPro theme by: Design Disease