San Francisco, March 2 (AP): Internet giant Yahoo! Inc is adopting a new system for indexing web pages that will charge businesses to include more material currently unlisted in its online search engine, marking the first volley in a duel with its former ally Google Inc.

Sunnyvale-based Yahoo is touting the approach, scheduled to be announced on Tuesday, as a practical way to assure its search engine captures more of the so-called “deep web” — the billions of pages that aren’t found during periodic crawls of the internet. The method, often called “paid inclusion,” also will help Yahoo’s search engine keep better tabs on the most current material on a web page, company officials said.

More than 99 per cent of Yahoo’s search index will consist of web links that don’t pay fees, said Tim Cadogan, the company’s vice-president of search.

Search engine analysts generally applauded Yahoo’s move, saying it could open a rich new vein of content that’s lacking from all internet search engines.

But the fees required to participate in the programme are likely to raise worries about Yahoo creating an online caste system dividing the haves and have-nots of the internet.

To ease those concerns, Yahoo isn’t charging non-profit web sites to add unlisted links to its search engine. The non-profit sites initially participating in the new indexing system include national public radio and the library of congress.

While Yahoo’s index will continue to include web sites that don’t pay the fees, there’s no guarantee on how frequently those destinations will be visited, Cadogan said. The fees won’t buy web sites a higher ranking in Yahoo’s non-commercial search results, he added.

The fees under Yahoo’s “content acquisition program” will be based on the size of the participating web sites, how many unlisted links are submitted and how frequently the links are clicked on by the users of Yahoo’s search engine.

Yahoo is counting on the programme to give it an advantage over Google as it vies to supplant its rival as the web’s most popular search engine. Yahoo licensed Google’s search engine for more than three-and-a-half years, but started to cut ties with its former partner two weeks ago, vowing to introduce better ways to explore the web.

Google has built the web’s largest search engine index, spanning 4.28 billion pages without charging fees to be included. Yahoo says its index contains “several billion” web pages, but won’t provide specifics.

Google co-founder Larry Page called Yahoo’s new system “a pretty bad thing to do. There are plenty of profits to go around in search engines to find ways to improve the user experience without charging fees to do it”.

Like Google and other major search engines, Yahoo has been using its search results page to display text-based ads that are tied to search requests. But these advertising listings are labelled as “sponsored results” and separated from the results generated through algorithmic formulas designed to provide an objective analysis.