tdog.blog

Hi there! Welcome to our blog. Don't forget to sign up for our free RSS feed. We Triple Dog Dare Ya! And thanks for visiting!

Building a custom search engine for a web-based application (particularly a CMS) can be a daunting experience the first time around. No matter what approach you take, what platform you’re using, or what kinds of users you have, you need to keep the Three Laws of Search Engine Usability in sight.

(With my apologies to Dr. Asimov)

Rule 1. The search engine must be fast.
Rule 2. The search engine results must be easily differentiated to the user–they need to be able to get to the information they need quickly.
Rule 3. Never do anything with Rule 2 that adversely affects Rule 1, or vice versa.

Rule #1 is pretty self-explanatory. If your search engine is slow to produce results, people won’t use it. Why would it be slow? Are you physically searching through text or HTML files or using xpath queries through XML to get what you need? Then use some kind of text-based index that is created every few hours by a cron job.

This index would essentially be a reverse dictionary. For every word found in your files (minus stop words like the/and/a/what/who/when/where/how etc) you would create a listing that would map word to each file it appears in, like this:

Lincoln index.html,presidents.html,lincolnbio.html
Washington index.html,presidents.html,washbio.html

Then your search engine could parse the index instead of all the files. The good news? This method is speedier. The bad news? Your index is only as up to date as the last time it was run. There may be a lag lasting many hours.

If you are using a database query and its slow, think about indexing your most frequently accessed tables. On mySQL and other database products, you can turn on query caching and store relevate information in RAM.

Rule #2 is a bit more subtle. Instead of returning a pile of search results, each of which undoubtly contain a link, you have to provide more context and information design elements to help users make sense of the information provided by the search engine.

For example, you may want to bold the link to the document in question. Good first step. You may also want to display meta keywords, meta description, or the page title as well. You will also likely want to number your search results and provide plenty of space before and after each item in the result list to boost visibility.

Rule #3 is even more subtle than Rule #1 or Rule #2. Let’s say you have a fairly fast search engine, and you decide to use an icon next to the link for each search result. You want users of the site to start asociating PDF downloads with an acrobat logo or press releases with another type of logo. Great idea!

Just don’t go overboard with #2 or you’ll start affecting #1. What do I mean? Let’s stick with the same explamble. Don’t use rollover effects on your image or an aimated image. Don’t use a very large image. Either one of these will add server load and processing time if the search result is long or complicated.

On the other hand, don’t just say, “We’re goin’ for speed!” and just dump out a bunch of links. The folks who want to use the search are doing it for one of just a few reasons:

* They are looking for a specific document
* They are looking for a particular class of documents (all the press releases from 2004)
* They are looking for documents that live in different regions of the site (all PPT presentations on Disaster Recovery regardless of whether they live under Products, Services, or Consulting).

Bookmark:

Comments(0)

When we’re launching and/or monitoring a customer web site, it’s important to have good information about server performance at our fingertips. We’ve developed a handy, short shell script that helps us monitor server load and the number of processes that are present for mysql and httpd.

Here is the script:

#!/bin/sh
date
uptime
echo -n "mysqld: "
ps aux | grep mysqld | wc -l
echo -n "apache: "
ps aux | grep httpd | wc -l


Save this script as checkup and make sure you run chmod +x on it. From the command line, you can invoke it as ./checkup and get the following output:

Wed Sep 21 18:31:04 CDT 2005
6:31pm up 173 days, 4:44, 1 user, load average: 0.72, 0.81, 0.73
mysqld: 16
apache: 33


As you can see, the script prints out the current date, the results of uptime (which includes server load averages for the last 1, 5, and 15 minutes), and the number of mysqld and apache processes. Many times, this simple script is enough for our admins to get an overall picture of the server’s health.

Something else we do? We can redirect the output of this script to a log file and then email that log file once a day to other interested parties.

Bookmark:

Comments(0)

We’ve been involved with a lot of projects here at Triple Dog Dare Media in the past 4.5 years. They’ve ranged from fixing someone’s battered PayPal script (1.25 hours) to building an enterprise-level CMS for a Fortune 1000 client (8 months) and lots of stuff in between.

We’ve worked for clients that are amazingly communicative, transparent really, and others that were like brick walls (and even one or two we had to fire because we weren’t really quite sure if what they were doing was exactly legal). We’ve worked in the non-profit and government sectors, and industries as far-flung as manufacturing, consulting, travel, sales, software publishing, and contracting services.

One of our first clients, Dave Claunch of Liaison Resources, is one of our heroes. He taught us a lot about contracts, creating deliverables that everyone could live with, and the details of negotiation. Sitting at the table with him was like getting your butt beat by a real poker pro in Vegas–you didn’t mind the beatings because you were learning so much.

Others, like Mary McCommon of IBM and Renee Delaune of Delaune & Associates also come to mind as people we’ve learned a lot from. These two ladies have moxie, business savvy, and lots of years on the mean streets of Corporate America. Without their guidance we’d never have made it out of toddlerhood as a company.

We won’t take the time to mention the bad stuff specifically–and no, just because we haven’t mentioned you with the good, it doesn’t mean you were an example of bad stuff!–but I thought it might be helpful to go over some of the ways a project can fail. I even asked my staff to help me compile the list.

And yes, each of these are for real–they’ve actually happened to us on real projects. Not all of them caused total project failure, but they certainly contributed to more than anyone’s fair share of trouble.

So here goes….

• Don’t divulge budgets, deadlines, or complete names of stakeholders–or keep changing this information without telling us.
• Set a deadline of “whenever”.
• Set a deadline that is ridiculously close and arbitrary (i.e., build an extranet portal in time for a sales meeting in 10 days).
• Tell us we don’t have time to fully flesh out functional/technical requirements, but do complain when applications aren’t built the way you wanted them.
• Neglect to tell us if certain minor stakeholders (like the CEO’s spouse) have veto authority on some aspect of the project.
• Opt for the all-or-nothing moonshot instead of smaller, less-risky, phased deliverables.
• Remove QA, usability testing, and documentation lines items from the budget.
• Communicate with us piecemeal (hundreds of little emails, IM notes, hallway conversations) and never in structured meetings or with lists/agendas.
• Avoid user testing, even with proxy users from your own company before releasing software to the target market.
• Insist that prototype only be viewed/tested by executive committee and not the actual users of the product.
• Insist on cutting project management from our side because you have a project manager from marketing assigned on your side.
• Insist on not using a version control system and backups for the project.
• Insist on announcing to the entire marketplace what your intentions are for the project even before we’ve had a kick-off meeting.
• Set a kick-off meeting but don’t invite key stakeholders.

Bookmark:

Comments(0)

The good folks at CIO magazine have published an interesting article that provides a nice background on the field of KM.

Here is a snippet from their intro:

Think of a golf caddie as a simplified example of a knowledge worker. Good caddies do more than carry clubs and track down wayward balls. When asked, a good caddie will give advice to golfers, such as, “The wind makes the ninth hole play 15 yards longer.” Accurate advice may lead to a bigger tip at the end of the day. On the flip side, the golfer — having derived a benefit from the caddie’s advice — may be more likely to play that course again. If a good caddie is willing to share what he knows with other caddies, then they all may eventually earn bigger tips. How would KM work to make this happen? The caddie master may decide to reward caddies for sharing their tips by offering them credits for pro shop merchandise. Once the best advice is collected, the course manager would publish the information in notebooks (or make it available on PDAs), and distribute them to all the caddies. The end result of a well-designed KM program is that everyone wins. In this case, caddies get bigger tips and deals on merchandise, golfers play better because they benefit from the collective experience of caddies, and the course owners win because better scores lead to more repeat business.

Bookmark:

Comments(0)

Everyone wants their web site to stand out in a Google search. After all, Google logs millions of requests every day, and its technology is being used by more and more high-profile search engines (like Yahoo). Being in the top 10 results on Google could mean a lot more traffic to your site.

One sure way of rising to the top of the pile is to actually shell out some dough to the good folks at Google. This technique is called search engine placement, and it can get you the preferential treatment you want. However, there are other FREE alternatives to achieving the same goal, and they all fall under the term search engine optimization (or SEO).

The key to a good search engine optimization campaign is to understand how a search engine thinks. I’m only going to talk about Google because every search engine has a slightly different temperament. Basically, Google’s search engine tries to determine how relevant your particular site is to the search term. It does this by trying to determine how many other sites have linked back to you, and by comparing your site’s metadata to the search term, and finally, by comparing your site’s metadata with your web copy.

Although that all sounds pretty complicated, it really isn’t. Here are a few steps to increase your Google rankings (please note that these techniques can take weeks and months to take effect):

1. Keep your metadata tight and to the point. Keep your HTML titles down to 10 or 12 words. Don’t use more than 8 or 10 descriptive keywords. Why is this? Because too many keywords will in effect lower your ranking–this is because the algorithm won’t believe (if I can humanize it a bit) that your web page can actually be about so many topics.

2. Align your content with your metadata. Make sure that you repeat your keywords in your headers (particularly your first H1 tag), in the first paragraph of body copy, and link text. Also, you can place your keywords inside HTML comments and in the ALT attributes of images (yes, I know, these are normally used to aid the blind, but hey, we need search engine visibility here!). Doing these things will help convince the algorithm that your page has been rightly described.

3. Get others to link to you. The more sites that link to you, the better off you’ll be. Don’t limit yourself to simple link exchanges, either. Whenever you write an article for an online zine or publication, make sure that your bio contains a link back to your site. Always have your URL in your sig when you post to online forums.

Over time, these efforts will raise your visibility. Good luck!

Bookmark:

Comments(0)