Jesper's dev blog

Adding search to our websites

Jesper Høy's photo
Jesper Høy
·

2 min read

Having recently moved more content (news and knowledge base) back to our own web-sites (used to be hosted on separate support sites), I have been looking for a way to also add a search function.

The popular "Google Site Search" service was recently shut down, and I haven't been able to find any other good "cloud based" alternatives - at least not at a reasonable cost.

I briefly tested AddSearch. This looks nice, is easy to install, and starts at $29/month. Unfortunately it is limited in how it can be customized and what kind of reports you get. You can only do a "full re-crawl" once per week - which sucks - especially during the initial setup phase where you are bound to mess up. And it turns out that we have more than 1000 pages to index, which brings the monthly cost up to $79.

I also looked into:

  • Site Search 360 - looks nice, but appears to be limited to 2 re-crawls per week even at their most expensive plan ($99).
  • Cludo - starts at $199/month.
  • Swiftype - starts at $299/month.
  • Algolia - looks like this doesn't come with a web-crawler unless you opt for the $2,500/month "Business" plan.
  • Amazon CloudSearch / Bing Custom Search - as far as I can tell, these services don't come with a web-crawler.

I am disappointed that something this fundamental doesn't exist, as a basic cloud service, at a modest price in 2017 (we have been searching on websites for +20 years now).

So I gave up on "the cloud" and used an old friend - Wrensoft's Zoom Search Engine, a Windows software package which includes both a search engine and a web-crawler ($99 one time).

I programmed a simple frontend myself - forwarding each query to the Zoom Search Engine (using the cgi version), parsing the returned XML, and finally rendering HTML with Bootstrap styling etc.

This works great, I have it scheduled it to re-crawl every night, I can re-crawl on-demand any time I like, and crawling some 1200 pages only takes about a minute.

The result is now live at https://simpledns.plus/search and http://simplefailover.com/search

searchAlgolia