use Bing to search PMN

This area is for the discussion of what's new, what's on your mind, and general photographic topics. A place to meet, make comments on this site, and get the latest community news.

Moderators: rjlittlefield, ChrisR, Chris S., Pau

rjlittlefield
Site Admin
Posts: 23543
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

use Bing to search PMN

Post by rjlittlefield »

Summary: for searching photomacrography.net, the best external search engine right now is Bing .

Bing's coverage is far better than Google's coverage at this time.

The forum's own internal search engine is also quite good. It is at https://photomacrography.net/forum/search.php , reachable through the "Advanced search" cog icon next to the search box in upper right of most forum pages.

More details...

Some weeks ago, it was noted at viewtopic.php?p=272768#p272768 that PMN had essentially disappeared from Google, starting sometime after the forum upgrade last summer. I promised to investigate, and fairly quickly found that the problem extended to all the other search engines also.

After some thrashing around, I found what I believe was the root cause: there was a disastrous directive, <meta name="robots" content="noindex,nofollow">, in the default style template. That directive caused all the search engine indexes to slowly rot, starting at the time of update.
(The origin of the directive remains a mystery. I have no memory of sticking it in, and I cannot think why I would have stuck it in, but I cannot find any other source. So, I'm choosing to assume that I did it -- suffering from some combination of derangement and amnesia -- because for me that's far more comfortable than suggesting mischievous hackers or supernatural phenomena. Anyway...)

I removed the troublesome directive and also generated some sitemap files that specifically point to the roughly 45,000 URLs of old topics that we would like to have indexed.

Bing seems to be making progress pretty quickly. As of this moment, a query at bing.com of "site:www.photomacrography.net/forum" responds with "19,600 Results". Restricting the search to "site:www.photomacrography.net/forum bratcam" responds with "92 results", including the oldest such thread from 2009.

Google, in contrast, is proceeding at a snail's pace. At this moment "site:www.photomacrography.net/forum" says "About 3,230", and "site:www.photomacrography.net/forum bratcam" says only "About 24 results", none of those about the Bratcam itself.

I monitor the list of "online users" periodically. What I see is that the Bing bot visits frequently and is methodically "Reading topic in", while the Google bot visits less frequently and is spending its time mostly on "Index page".

I have no idea how to make the Google bot do its job any better. I have explicitly submitted the sitemap, which they acknowledged receiving but now seem to be ignoring. I have looked for, but not found, any management console settings that would restrict Google's crawl rate. I have also looked for ways to request a faster crawl, but find only statements that there aren't any of those.

If anybody with expertise in this area has suggestions for other things to check, I would of course be happy to learn about them.

--Rik

viktor j nilsson
Posts: 423
Joined: Fri Mar 01, 2013 1:43 am
Location: Lund, Sweden

Re: use Bing to search PMN

Post by viktor j nilsson »

Rik, I am very glad to see that you are working on a solution to this problem, and that you have found a temporary workaround. =D>

Lou Jost
Posts: 5933
Joined: Fri Sep 04, 2015 7:03 am
Location: Ecuador
Contact:

Re: use Bing to search PMN

Post by Lou Jost »

That's unfortunate; I had in the past found Google to work much better than the internal search feature. I can't get the internal engine to search for an exact word series; for example, a search for "Minolta enlarger" finds all posts with both the words "Minolta" and "enlarger" somewhere in them, instead of the few posts that contain the exact phrase "Minolta enlarger". Is there a search command for this?

rjlittlefield
Site Admin
Posts: 23543
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Re: use Bing to search PMN

Post by rjlittlefield »

The built-in "phpBB native" search facility only does unordered combinations.

However, phpBB now supports an alternative search backend, "MySQL fulltext". I have tested that on the demotest forum, and it seems to do what you want.

So, I have converted the real forum to use MySQL fulltext search also.

Your example of "Minolta enlarger" is informative but quite restrictive, since that phrase is found only in your current post.

But, a search for posts by rjlittlefield, containing diffraction blurring without the quotes, returns 74 replies after this post, and only 51 when quotes are added as in "diffraction blurring" .

A search for "diffraction blurring" AND ("underlying issue" OR "pretty happy") quickly finds those two posts (and now this one), so I am encouraged.

--Rik

Lou Jost
Posts: 5933
Joined: Fri Sep 04, 2015 7:03 am
Location: Ecuador
Contact:

Re: use Bing to search PMN

Post by Lou Jost »

Wow, that is excellent! Thanks for doing that upgrade, it will make it easier to search for topics which consist of common words in uncommon combinations.

NikonUser
Posts: 2693
Joined: Thu Sep 04, 2008 2:03 am
Location: southern New Brunswick, Canada

Re: use Bing to search PMN

Post by NikonUser »

make this sticky!
NU.
student of entomology
Quote – Holmes on ‘Entomology’
” I suppose you are an entomologist ? “
” Not quite so ambitious as that, sir. I should like to put my eyes on the individual entitled to that name.
No man can be truly called an entomologist,
sir; the subject is too vast for any single human intelligence to grasp.”
Oliver Wendell Holmes, Sr
The Poet at the Breakfast Table.

Nikon camera, lenses and objectives
Olympus microscope and objectives

Post Reply Previous topicNext topic