Every search engine follows certain basic rules in order to generate results from the web pages: crawling, indexing, and ranking (serving). These steps follow one after the other systematically. Ranking cannot take place without indexing and indexing cannot take place without crawling.
Crawling is the process in which mechanisms known as ‘web crawlers’ access the web pages that are publicly available. Indexing is the process in which a search engine reviews the content of every page and accumulates all the pieces of information it finds. Ranking is the process in which a search engine presents the most relevant results taken from its index in a chronological order after a user types a query.
A website’s internal linking structure is very crucial since crawlers use the links on the website to find other web pages. New websites, any changes made to the existing websites as well as the dead links are prioritized accordingly by the crawlers. The websites which are to be crawled, the frequency of crawling, as well as the number of pages which the search engine will fetch; are all determined by an automated process. The hosting capabilities including the bandwidth and the server resources also influence the crawling process.
This is where the crawl budget comes into play. Crawl budget denotes the number of web pages that a crawler is prepared to crawl in a certain period of time. The search engine establishes the crawl budget for any website automatically, by taking certain key factors into consideration, such as: the size of the website (bigger sites need more crawl budget), the server setup (page load time and the site performance), the frequency of updates (content that is updated regularly will be prioritized), and the links (dead links and the internal linking structure).
Search engines like Google have mechanisms that ensure that their crawlers visit a website only as frequently as is feasible for the website. Crawl rate limit is one such mechanism which enables Google to establish the crawl budget for any website. Crawl demand is another such mechanism which takes into account the demands from any specific URL from the index; so as to determine how active or passive the URL should be. URL popularity and its staleness are the two factors that help in determining the crawl demand.
An advertising manager needs to consider certain aspects before making an informed decision regarding the Google Ads account, such as: evaluating the performance based on location targeting, query matching, checking for changes made by the competitors, and looking for other possibilities for the account structure.
Let us look at each of these aspects in detail:
- Evaluating the performance based on location targeting: It is an excellent practice to choose the location of interest option in the Google Ads account settings, because it encompasses both the people living in the target location as well as the people only interested in the target location (but not living there). Also, evaluating the performance based on user location will enable the ad manager to pinpoint the areas which can boost the costs per acquisition (CPAs) and returns on ad spends (ROAS) for the business.
- Query matching: The ad manager should create a file in Google Sheets and use the pivot table function for arranging the search queries in the order in which they match the campaigns and ad groups that have been previously listed.
- Checking for changes made by the competitors: It is always good to keep an eye on the competition’s actions, both online (in the form of social media or anywhere else over the web) as well as in the market. The advertisers need to check for changes made (if any) by their competitors. They can accomplish this task by making use of the Auction Insights mechanism.
Looking for other possibilities for the account structure: The advertisers need to reassess their Google Ads account structure based on the business type of the account, its performance, as well as on the constraints (if any exist). Then they need to decide whether to look for other possibilities for the account structure (after the reassessing).