What is the intention of WebAggregate?

See a future of entrepreneurs, people supporting each other, that’s what i always wanted to create. How much can i do alone? This project is not my project, its a community project, where all working professionals like us will support students/freshers in their career.

Support structure -

There are 10+ Core leaders who had similar intention and they have lot of passion they dedicate their time in weekends and taking actions related to intention of WebAggregate.

There are Core Team & Mentors few people work in background process and few work in frontend process

Initially i was thinking we should connect individuals who are working, now i am thinking we should connect alumni smaller communities and connect them as one entity. With this intention i spoke to Mr Arun, who is our college senior, Mr. Shreedhar Sir HOD Computer Science department and finally Lakshya team who is doing similar work as part of WebAggregate.

It is a awesome experience to make a difference in others life, we always wanted some seniors to help us when we needed their support and also when we needed their mentoring, join hands by signing up in WebAggregate.

Its been fun working at Yahoo!, i always loved working on SEO and i got that opportunity for Yahoo! Health, Yahoo! Beta Cricket, Yahoo! weather in 2008, later i have been working as local security engineer for media group projects along with being Backend Engineer, and july 2009 now i moved to Yahoo! Finance.

I would like to acknowledge and thank  few colleagues -  Director of product management Don Chennavasin , Engineering Manager Raghu M, and Vivek Saraf Dedicated Security Engineer from who are always been source of inspiration.

Can we recover lost mysql root password or is there any way to change mysql root password?
When i was a fresher i have seen myself  when not able to access mysql, i used to uninstall and reinstall mysql server.
Well is there any better way to recover/change/reset mysql root password which is lost? Answer is yes, if you are a sudo user.

Step1: Stop the mysql process

sudo /etc/init.d/mysqld stop

step2: Start mysql in safe mode and skip the check against user privileges.

sudo /usr/bin/mysqld_safe –skip-grant-tables &

Step3: Since mysql is running in safe mode and we have skiped user privileges, now ogin to mysql without using password and use mysql database , update the password and flush the privileges.

mysql -u root
use mysql;
update user set password=PASSWORD(”newpassword”) where User=’root’;
flush privileges;

Step4: Let us test by logging in to mysql using the password. We need to stop the earlier started mysql process which is  running in safe mode with user priviliges check being skipped. Now start the mysql in normal mode and try logging in to mysql using the new password.

sudo /etc/init.d/mysqld stop
sudo /etc/init.d/mysqld start
mysql -u root -pnewpassword

In this article i will be writing about highlighting the keywords in search results and formating the output display format much similar to most search engines result page using the zend lucene search.

<?phprequire_once ‘Zend/Search/Lucene.php’;

$queryStr= "php";

$snapshotTextLength = 155;

$query = Zend_Search_Lucene_Search_QueryParser::parse($queryStr);

$index = Zend_Search_Lucene::open("/var/www/lucene-data/blog-index");

$results = $index->find($query);

echo "Index contains ".$index->count()." documents.\n\n";

if($index->count())

{

$count = 0;

displayResults($results, $snapshotTextLength);

}

// Format and display the search results

function displaySearchResults(&$results, $snapshotTextLength)

{

if(is_array($results) && count($results))

{

foreach ($results as $result)

{

$data[$count]["article_url"]         		= $result->url;

$data[$count]["article_title"]        		= $query->highlightMatches($result->title);

$data[$count]["article_description"]        = $query->highlightMatches($result->contents);

$data[$count]["article_created_date_time"]  = $result->postedDateTime;

$data[$count]["article_id"]             	= $result->articleId;

$count++;

// title of each article with URL as link

$searchResultsContent .= sprintf("%“, $data[$count][”article_url”], $data[$count][”article_title”]);

// snapshot of the description

$searchResultsContent .= sprintf(”%s”, substr($data[$count][”article_description”], 0, $snapshotTextLength));

// url

$searchResultsContent .= $data[$count][”article_url”];

// leave 2 lines after each search results

$searchResultsContent .= “<br> <br> <br>”;

}

}

else

{

$searchResultsContent = “No results found, try using different keywords”;

}

return $searchResultsContent;

}

?>

This program is similar to Zend Lucene Search - part3 - retrieving the indexed data , the only difference is i am formating the display format, the output of this program displays the output much similar to what you get in the search engine result page of google.com or search.yahoo.com

Related articles:
Zend Lucene Search - part1 - creating index
Zend Lucene Search - part2 - Real time indexing
Zend Lucene Search - part3 - retrieving the indexed data
Zend Lucene Search - part4 - Search Results Highlighting

But in Yahoo! i just see lot of very interesting tools been developed in PERL, i always thought why couldn’t it be coded in PHP? may be since i was from PHP programmer i always asked that question myself, but i see lot of my colleagues do lot of coding in PERL, pretty excited about it but not want to give up comfort zone, finally after 11 months i gave a try, i completed my first package done entirely coded in PERL.

I just entered the world of PERL, if you are a PHP programmer and feel PERL is not what you want to learn since you know PHP? I would recommend you give a try, you would love both PHP and PERL.

The best book for beginners in PERL is -
Learning PERL

Related links -
PERL
CPAN - The Comprehensive Perl Archive Network

So we should use robots.txt to disallow website search results pages crawling that don’t add much value for users coming from search engines. Its one of the quality guidelines Google mentions in webmaster guidelines. Its one of the important SEO checklist we should track.

Possible reasons:

1. Duplicate content -
   Search results holds a snippet of article title, short description and link to that article and in contrast we also have article page which has article title, article description. So if search bot crawls search results and individual article, it can potentially lead to duplicate content.
2. Door way pages -
   Search results acts like a doorway pages to individual articles, doorway pages are one of the cloaking techniques that should be avoided.
3. Search engine subsystem -
   Search engine as a whole strives to provide unique results, clicking on the links(indexed website search pages) in search engine results page which in turn takes to the website search results (running another search in website).

Example -
Disallow: /search?p=* in health.yahoo.com/robots.txt
Disallow :/search in http://search.yahoo.com/robots.txt
Disallow: /results in http://youtube.com/robots.txt

Following are the Search engine optiomization(SEO) checklist  related to robots.txt -

1. robots.txt http status code

Search bot (eg: googlebot) before crawling the website it will always requests robots.txt and understands the definition robots.txt and crawls the website locations which is allowed. So its always important for webmaster to check the http status code of the robots.txt of the site and make sure it returns http status code 200 or http status code 404.

Why is that so important to check the http status code?

Search bot before crawling requests for robots.txt-

If the http status code of 200 is returned, it reads and crawls the locations of website which is allowed.

If the http status code of 404 is returned, search bot goes ahead with its job with no restriction on website crawling.

If the page takes lot of time and if there is no response code returns, search bot waits and after sometime it skips crawling because it always respects robots.txt and this can adversely affect the crawling of our website.

2. URLs restricted by robots.txt

Consider the impact of following robots.txt definition

User-agent: *
Disallow: *

It blocks all search bots to crawl the entire website, we should make sure we block only those areas which block.

How do you inform search engines that your website existed and inform them to index your website?

When i started working on Search Engine Optimization ( SEO ) for 3 ecommerce sites in 2006, this was the first question i had in mind.

Following are the ways of getting your website indexed by search engines -

• Submit site to dmoz -
  Go to dmoz - open directory project and submit site to dmoz , search engines (like google, yahoo etc.) get the data from dmoz - open directory and they crawl the websites.
• Submit site to google -
  Go to google search and submit site to google search engine, this will inform google to crawl your website.
• Submit site to yahoo -
  Go to yahoo search and submit site to yahoo search, this will inform yahoo to crawl your website
• Submit site to live -
  Go to live search and submit site to live search, this will inform live search to crawl your website
• Create sitemap and ping search engines
• Mention the sitemap path in robots.txt
  

Above links leads to the same page, but it differs with www.
But search engine treats both links are different, i have seen in few cases where we link many a times we ignore www. and in some cases we do include www. in the links.

So what are the impacts?

1. Search engines keep both the versions of the URLs, when people click on search engine results links which leads to our site with different versions of these URLs, it will drastically affect the page rank and traffic.
2. These URLs look like different documents to crawlers and create excessive crawling on our website.

How do we instruct search engine to treat both the URL’s as same, Google webmasters tool has a option to set the preferred domain

So whats the advantage of set preferred domain ? If i set my preferred domain as ganeshhs.com and next time if Google comes and crawls my website, and if it finds any link starting with www.ganeshhs.com it will follow it as ganeshhs.com and when Google displays my website links in search results it will show the links as ganeshhs.com

It also helps us to fix the external site referrals, few guys started provide links to my website, if suppose their referral link is http://www.ganeshhs.com/category/search-engine-optimization-seo where as my actual article URL was http://ganeshhs.com/category/search-engine-optimization-seo and when google crawls our website through that referral link it will keep the right version of domain what we preferred.

In some cases, we come across situations where we keep links to external sites. But what are the impacts of this?

1. Part of page rank is shared to external website -
   When we link to other websites, our part of our website page rank is shared to
   those external sites, and we may end up sending the search engine crawlers to other side.
2. Leading Search Engine Crawlers to crawl external website -
   Crawler entered our website to crawl more pages, it will help us to have more indexes in Search Engines, but what did we end up keeping external links, we created a way to Crawler to leave our website and crawl the external websites.

We have to keep external links, but how do we prevent the above scenario -