Okay, so originally, keyword catcher was written for people who hate wasting their time surfing to weblogs or Web stats or Webalizer via their cPanel (or, just plain forget!) to extract keywords and number of visitors and what have we more, and basically have the code scrape the keywords the very instant that they arrived and make this visible immediately or have this information sent to them via e-mail.

Many people still don’t know how to monetize the information their own webhosts provide from the traffic they generate.

This was done to avoid the usual delay filters that are applied by most domain stats compilers so as not to overload the servers. Most of us know that when we are looking at our cpanel webalizer, we can get more recent information by refreshing the page. But this information has been compiled so engineers can use it, not web marketers.

Web marketers need this information to be presented so that they can spot opportunities for commercial expoitation easily and quickly.

If you have a product that is catering to a very specific public and you find out that your public is using a number of keywords to find your products, then you can concentrate on the keywords that are profitable and gain the advantage of not wasting any time with those keywords that are not useful.

As our work progressed, we found out that we had written a little piece of code that could do a lot more.

Basically, keyword catcher will be able to perform a number of tasks for you:

• provide you with immediate knowledge about the precise keyword queries performed by people who landed on your site
• keyword catcher allows you to know immediately where your webpage was ranked by the search engine that provided it to the visitor that is surfing your domain
• Provide you with immediate knowledge of which sites are referring visitors to your domain

When you upload the extract code to your webpage, keyword catcher will start scraping information from your website logs immediately and will present them to you in a way that you will very easily be able to spot valuable information within minutes of setting it up.

It is sometimes of crucial importance that we find out and record if search engines are providing us with visitors and if these visitors landed on our site thanks to a specific keyword query.

On top of that, search engines and weblogs provide our hosts with a number of information tags we can go read and rearrange in such a way that the received information can be interpreted to become important and valuable immediately.

For example:

In the event you are setting up a PPC campaign to test any number of new keywords.

You can ask keyword catcher to send you an e-mail warning you that a certain search term has brought a number of visitors to your website a specific number of times.

Knowing how costs add up very quickly in adwords, you could know from experience that when a keyword has been queried a number of times, for example 100 times, and this has not produced any sales, that it is time to improve upon or end the experiment because you have confirmed that this specific keyword in this specific campaign is not profitable.

The advantage gained by this feature is that you can get an immediate warning to your e-mail box.

Let’s take the classic dog training example:

Imagine you want to test the keyword “labrador training” but that you want to go to your PPC account and stop the test after 100 people have searched for this term and landed on your page to check results.

You could decide to exclude the broad term “labrador training” in your PPC campaign because you will see Google sending you traffic for “Where to find labrador training gear” or “labrador training china” 0r “labrador training school Ohio” and these people click on your Paid Per Click Ad out of pure curiosity, not real interest.

With Keyword Catcher this becomes easy: we simply go to the notifier and ask it to send us an e-mail when it finds ninety queries for the keyword.

At the same time keyword catcher will scrape a number of very interesting information bits from our stats compilers and make them available to us immediately.

Conveniently, we can centralize all the information for all our domains under a single account.

By logging in into our account, we can check out whether visits are still being driven to our domains by the various channels which we have chosen to exploit, such as e-zine articles, Google blogs, Squidoo Pages, Hub Pages, PPC campaigns, and so forth.

• exclude tire kicker keywords rapidly
• excludes negative keywords as soon as you see them
• exclude non-profitable keywords as soon as you identify them accoring to your own criteria
• spot the buyer keywords quickly
• extract the unique keywords queries that come only to your site

a) The number of visitors that came to your website?

b) The number of search engines that crawled your pages?

c) Keywords that people use search via search engines to land on your web sites’ pages?

d) The countries these visitors came from?

e) The bandwidth that was used during a period of time?

Did you know that these server side statistics programs are usually provided at no cost whatsoever as an extra service to subscribing customers and that most service providers provide more than one type of statistics program?

My service provider, for instance, makes available three types of statistics gathering software programs which I can open and read via my user interface so that I have three different sources I can cross-reference.

They are:

Awstats
Advanced Website Statistics
Webalizer

These statistics pages are of enormous importance to any website owners who wishes to keep track of a number of parameters that can help them develop their online business. Knowing which keyword their visitors typed can be of paramount importance as this can help with finding inspiration for fresh content.

About the disadvantages:

a) The information statistics results pages provide is not easy to read, it would seem that these have been designed by and for engineers, so that it really takes some getting used to.

b) Distilling useful information from statistics web pages can be a boring and time-consuming task.

c) Exporting and filtering or discriminating results is often complicated or even not available.

d) The results these servers collect are not always complete. Sometimes, a web server manager might decide that for certain periods of time, for example, while some or other update task is running, that the statistics software needs to be shut off. During this period the server is not collecting any information and you will not even know about it.

e) There is usually a 24 hour time lag between an event, such as a visit to a specific page, and a retrievable result.

About the 24 hour time lag:

This has a historic origin. In the early days, when random access memory (RAM) was extremely expensive and hard to come by, software engineers had to make do and live with the fact that the space available to provide a computer with instructions was very limited. On the server side of things, it was easy enough to install the statistics gathering software programs, but they were considered resources hogs.

To limit the amount of resources these programs would use on a web server, someone had the bright idea to limit less essential software programs to update only during low traffic periods. It was rapidly known by users that statistics became available usually a day later. This set a precedent and created a habit that still persists today.

The fact is that computing power as enormously progressed and that waiting a whole day for stats to appear as become completely redundant. Nowadays servers are so powerful that they could provide you with real-time data.

You could for instance chose to install an application that provides you with real-time useful feedback from your web server. One such application is named Keyword Catcher. This is a server side PHP script that will show you the queries that visitors typed into their browsers via Google, Yahoo, Bing or any other major search engine. The search engine results page produced a link to your website and the visitor clicked on it to land on one of your websites’ pages.

Keyword Catcher will now be able to reproduce the exact same page that your visitor was presented by the search engine. This allows you to see which page rank your web page had at the time of the query. That in turn gives a website owner enormously useful feedback on how the search engines perceive his web pages, allowing him to take corrective measures or make use of opportunities.

As things stand today only a minority of people is aware of the enormous potential of server side statistics or the inherent lucrative use this information gathering can provide in terms of business intelligence. Only very large corporations have the budgets to develop and maintain the software needed to distil useful business intelligence. Keyword Catcher is a first step in making available strategic information to website and small business owners who feel the need to know more about people visiting their web sites’ pages.

There are two ways you can look at what a bounce rate tells you.

It is the number, expressed in a percentage of traffic to your website, that either is:

1) a visit to any of your web pages by a person who does not visit any other page.

or

2) a visit to any of your web pages by a person who does not stay longer than 10 seconds.

The trouble with both these definitions is that they are not clear enough. In both cases the information they provide is spectacular but the depth needed to interpret this is sorely lacking.

Indeed both definitions leave us wondering what the visitor to a webpage actually did.

On top of that, we are left to wonder if the visitor to our webpage might have lingered for a longer period than 10 seconds. There are many instances in which this could be the case.

The only certainty we really have is knowing that the surfer has seen only one page and no other.

There is a hack around this and it has the added advantage that you will get better information from Google analytics.

Under this line in your urchin code:

pageTracker._trackPageview();

you add:

setTimeout(‘pageTracker._trackEvent(\’NoBounce\’, \’NoBounce\’, \’Over 10 seconds\’)',10000);

The “10000″ number refers to the number of milliseconds you want to wait before triggering the above code, so if you want it to trigger after 12 seconds, you need to set this number to “12000″.

Now you have set your tracking code to record all visits that last longer than 10 seconds.

This allows you to get the precise number of visits that do not need to be considered as a “bounce” according to definition number two by providing the precise number of visits that really stay less than 10 seconds. It will give you a better insight into visitor engagement.

The power of real-time data is that it brings you much closer to what is happening out there right now. This allows your intuition to follow through on small leads that might lead to big results.

It’s also very stimulating to look at real-time data streams, which is why the latest real-time keyword content finding source from Google is a real gem to add to the keywords tools you might want to consider when you are short of inspiration.

When you use the following search query string in your browser, Google will open up a web page in your browser that will present you with a real-time published data stream related to the keyword you type into the search box.

Google’s Real Time Data Stream for the word “Apple”

http://www.google.com/search?esrch=RTSearch&tbs=rltm%3A1&tbo=u&hl=en&q=apple

So now, when you publish your content or article, you can keep an eye on this page to see when Google’s spiders will have crawled your server and picked it up. personally, I wouldn’t waste too much time staring at the page for it might be a while before your title appears.

I tested it with this page. Just before publishing it, I typed in the main keyword I targeted and I kept my beady eye on the data stream for a couple of minutes. At first I thought that I had not used the correct keyword. So I opened a new page with another term: “Google”. (It is in the title!)…

Then I thought: “Maybe the spiders only will come once a week because I set the “All in one SEO plugin in Wordpress to “weekly” for blog posts. ” I also noticed that the real time data included tweets, so I tweeted this post.  I did not see the post appear, so I guess that a limited amount of servers are being tracked for this and I was not about to waste any more time to find out.

On the other hand, if you want to check whether a certain keyword really has tough competition, it is safe to say there will be many publications related to this keyword. As an example, you could try to type in “Apple” and “diet” and compare the number of publications that appear, for example, in a time span of one hour.

Last I looked, “diet” had about a rate of 14 to 15 content items per hour while “Apple” had a rate of about 600 published items per hour! Mind you, my point of view is European, which means this publishing rate is due in late evening to early night. It stands to reason that the frequency of published items will vary according to location and time of day.

You will probably notice that a rate of 14 publications per hour provides for a rather static data stream while a rate of 600 publications per hour provide for a very lively and and quickly paced update rate.

Also please remember that this query string relates to a page that is currently in beta phase of testing and that Google is still tweaking it, so that by the time you read this, it is quite possible that the page will behave differently than the way I described it.

In any case, this is yet another very interesting and powerful research tool that will provide you with an important indication as to the intensity of publications for certain keywords.

The hard reality today is that our servers and websites receive and exchange information which we can see and interpret but just don’t use to improve our market performance. There are applications and software programs available to help with this, but for now, most businesses apparently choose to largely ignore the transparency this precious source of information can provide.

Yet, it goes much deeper that that. There is a chasm between traditional business and modern technology that most are aware of but probably choose not to acknowledge. After all, if things work as is, why stir in the mud to trouble the waters? If we all know that “knowledge is power”, there are but precious few among us who choose to look at the possibilities. Isn’t there a saying that goes: “never change a winning team” ?

I quote from the white paper that Eric T. Peterson, CEO and Principal Consultant at Web Analytics Demystified published in October 2009, “The Coming Revolution in Web Analytics” wrote, sponsored by SAS.

Today, a majority of companies are dramatically under-invested in analyzing data flowing from digital channels. Even when business managers have committed money to measurement technology, they usually fail to apply commensurate resources and effort to make the technology work for their business.

Instead, most organizations focus too much on generating reports and too little on producing true insights and recommendations, opting for what is easy, not for what is valuable to the business.

This to me, literally screams opportunity. Here we have a fast rising market and an obvious need. It is about middle management being able to do a part of their jobs properly, that is, not only generating reports but including meaningful insights as to what these reports are actually showing and saying.

It boils down to allowing those who can see to contribute to the “kaizen” principle, you know, that alien notion of continual improvement, that so many like talking about and so few put into actual practice, by making the information available and acting upon it.

The other hard fact here is that nobody likes bad news or whistle blowers. When we start reading the signs, chances are they will probably also point to areas where improvement is long overdue or to “hot spots”, segments of the organization that are past their failing point and generating costs.

The fact that sometimes this is exactly the situation that leaders want to maintain in some areas, be it either for political, strategic or tactical reasons, will surprise only the naive. The point is, that when this situation arises, leaders will want anlytics to be shut down, which will be a hard thing to do. Analytics bring more transparency and that needs to be carefully controlled for obvious reasons.

This atavistic fact is hard wired into our subconcious minds and the wisdom of the masses reflects this through sayings like “don’t kill the messenger” or “throwing out the baby with the tubwater”. With analytics, this is in fact what usually happens. People read reports and look without seeing. The exceptional few who can see are cautioned to be utterly discrete.

Mr. Peterson, who writes: “opting for what is easy, not for what is valuable to the business“, forgets to mention that usually, those who are responsible for this work get a very limited mandate, the unspoken cardinal rule being “mind your own business”, to which I would add:…”or lose your job”…

We stand at the eve of the third revolution in web analytics. Current evolutions like cloud computing, cheap large data storage facilities and the integration of mobile technologies line phone and GPS are opening new possibilities.

The availability of this data, coupled with an organization’s increasing willingness to mine the information for insights, is leading to an analytical revolution. Soon companies that historically relied on small samples, third-party data and gut feel will have access to huge data stores of information about their customers.

This data will span the analog and digital realm and, in Web Analytics Demystified’s opinion, power new businesses, new business models and entirely new relationships between businesses and the customers that support them.

We can speculate about making available free mobile phones for the right to monitor the phone user’s movements. One can imagine free access to the Internet in exchange for the right to monitor the browser’s page visits and even imagine important discounts to allow the seller to share the client’s data that is collected during the sales process. We can add the twist of the privacy issues this will create.

Let us not deny that this situation already exists!

Google is most visible among a handful of online companies that have learned that data about the transaction is as valuable as the transaction itself. And while some may respond that “Google is limited in what they can do with our information,” the reality is they are not. If Google wants to start using our data in a way they believe will create more value for Google, all they need to do is offer us something in return.

We have trained a generation to react on the Internet practically only to what is free. In the “make money online” segment, the proof of this is abundant. These days, the only offer that still work are free give aways coupled to an OTO or a free membership deal on trial with thousands of bonuses.

There are a number of search terms that come up when we think of our website’s performance and how we can track,  analyze or check up on how well a website is doing and there is a very active market that offers a diversity of solutions to choose from.

The most common are: stats, analytics, urchin, counter, hits, traffic, visits and track.

In Keyword Catcher’s case, we have tackled the question from the angle of the “KISS” principle. (keep it surprisingly simple)

There are but few among us who like or understand statistics mathematics, the idea alone conjures images of abstract complexity which most prefer to avoid or even flee and it is a hard learning curve climb.

Yet the question remains: what can one or should one do with webstats and is tracking analytics useful? A large percentage of website owners, until recently, to my great shame, I was among their numbers,  never look at them or when they do, can’t make heads or tails from the data that has been gathered.

Is it any wonder? No. The site statistics most get served by their hosting providers are a tough read, keeping in mind that they were probably designed by software engineers for their peers.

Most website statistics tracker software providers silently thank the engineers for this because it created a need (a market) to which they could provide an answer (a product).

Many providers agree that real time information is crucial these days. A majority argues that knowing about keywords (search terms or queries that visitors type into their search engine page query box) that attracted visitors to one’s website is of paramount importance and that knowing about referring websites is vital.

Indeed, on both counts they are right. If you want to find something on the Internet, and chances are you will find what you are looking for, you still have to know which search term to use when you “Google” it.

But more importantly, it is true for all website owners who set up their websites to help them generate an income. The first rule on the Internet is: no traffic, no business. Attracting visitors is the name of the game. Keyword Catcher can help with this by providing the strategic data that is needed to implement more efficient traffic generating tactics.

Keyword Catcher is a software that can be installed on the website server which will allow its owner to see both the keywords and the referring sites that have attracted a visit to one of the published pages in real time.

It does so by way of an easy to use and read dashboard while ensuring total privacy. Keyword Catcher will simplify monitoring statistics and log file analysis while mining and presenting relevant data in an easily manageable format.