How Do Home Computer Users Browse the Web? How Do Home Computer Users Browse the Web?

Most often, studies of user web browsing employ large-scale data collection to capture the web requests made by thousands of internet users. These studies frequently rely on server-side logging, collaboration with technology companies, or third-party data aggregators to compile such large datasets. Once collected, user behavior is most commonly modeled as a clickstream: a series of web requests over time. Clickstream analysis has been used to study a variety of topics related to web browsing including user security [4, 46], Sybil detection [56], user demographic prediction [21], and purchase behavior [28]. In addition, there have been a series of studies conducted using clickstreams in specific browsing contexts such as social media websites [2, 57, 58] and search engines [22, 37]. More closely related to our study, several works have leveraged clickstream models to examine user behavior more broadly. These studies assess the types of websites a user visits, identify patterns in user navigation, and measure the time users spend on web pages, also known as dwell time [1, 26, 29, 32].

However, clickstream studies, which typically employ server-side measurements, suffer from three primary limitations. First, Vassio et al. [54] estimated that as little as 2% of web requests captured in clickstream models are user initiated. The overwhelming majority of web requests are generated automatically to load dynamic content and buffer multimedia. With the pervasiveness of streaming video and dynamic web content in modern websites, the noise that automatic requests introduce into clickstream models makes it difficult to distinguish what is actual user behavior. Second, server-side measurements may miss when users return to a recently visited web page since these pages are often cached by the browser on the client side. Given that users frequently return to pages they have recently visited, clickstreams may miss a substantial amount of user navigation [29, 48]. Third, although clickstream models capture user actions that result in the generation of a web request, they do not reflect when users create, switch, or remove tabs within their browser. Since the introduction of browser tabs in the late 1990s, tab use has grown in popularity. By 2010, browsing sessions that used multiple tabs exceeded those that did not [23]. Although a clickstream model might show a user staying on a single web page until the next web request, the user may have actually been viewing several different web pages in other tabs during this period of time. As a result, clickstreams do not fully capture how users navigate the web, and subsequently their measurement of dwell time is inaccurate.

Of the previous studies mentioned, only Lehmann et al. [29] acknowledged this limitation and attempted to account for tab navigation by inferring tab changes from server-side logs. Although this represents an improvement over basic clickstream models, the proposed solution was unable to distinguish tab changes from back button navigations and could not tell if more than one tab change had occurred between requests.

In contrast, client-side studies of user behavior do not face the same limitations as server-side measurements. This is because these studies can log user activity directly rather than interpreting it from captured web requests. As a result, user-based studies have served as the foundation of our web browsing knowledge, providing insightful snapshots of user behavior between 1994 and 2011 [6, 14, 36, 43, 48, 63]. However, due to both the expense and technical challenges of scaling this type of study, there are relatively few and those that exist draw on a small number of participants.

To fill this gap, the SBO was developed to provide client-side data collection from a large panel of users [17]. The SBO initiative builds on the HomeNet project, an early work building panels of internet users [25]. The data collected through the SBO project has been used to provide insight into a variety of user behaviors including the use of passwords and private browsing [18, 45]. Another contemporary project similar to the SBO has been used to study malware attacks [31]. However, this recent study has not examined user browsing behavior more broadly. For this study, we obtained a subset of the SBO data and analyzed it to better understand browsing behavior generally and provide a much-needed update to previous user-based browsing studies.

Given that the most recent user-based browsing study was conducted nearly a decade ago, it is important to assess how user navigation has changed over time. Previous user-based studies of browsing behavior have focused on the frequency of various navigational actions, such as clicking a link or submitting a form, and high-level browsing metrics, like the number of web pages or the breadth of websites visited [6, 36, 43, 48, 63]. In particular, several of these studies’ primary aim has been to measure the revisitation rate: the frequency that a user returns to web pages they have previously visited compared to those they have not [36, 43, 48, 63]. These studies, conducted between 1994 and 2011, provide a series of valuable snapshots of user behavior. As such, the first aim of this study is to compare our measurements from the SBO to that of previous work, identifying changes and trends over time.

One pattern that previous work has already identified is the increase in the use of multiple browser tabs to switch between web pages [14, 23, 29]. This behavior, known as tabbed or parallel browsing, has been studied in the context revisitation rates [63] and search behavior [22, 23]. However, the effect that tabbed browsing has on user browsing generally and the way in which it is measured is not well understood [29]. This gap in knowledge motivated our analysis of tabbed browsing behavior among SBO users and our quantification of the error inherent in web measurements that fail to account for the use of multiple browser tabs.

In addition, Obendorf et al. [43] present an argument against the concept of an average internet user. Although this argument is supported by some subsequent findings outside of the context of user browsing, it is in direct tension with the continued development of software and design of research studies that are predicated on the assumption that an average user exists [15]. Given this contradiction, and the changing nature of user browsing behavior, we provide an updated assessment of the average user as a concept and evaluate clustering browsing behavior into user archetypes.

Large-scale server-side browsing studies have examined user behavior across a variety of different contexts including the use of search engines [8, 22, 23] and social media sites [2, 32]. However, these studies do not cover how users browse the internet as a whole and the many different types of websites it contains. Although user-based studies paint a broader picture of web behavior, their focus on high-level metrics like revisitation rates do not provide enough actionable information for researchers and practitioners. This methodological gap warrants additional study of user browsing behavior using an exploratory lens. Leveraging the advantages of the SBO, we contribute a detailed picture of how users spend time on the internet and how they navigate between sites across the web.

In particular, the time that users spend on low-visited websites is of great interest to researchers because it has been demonstrated that these sites have a higher likelihood of containing risky or malicious content [24, 33, 35, 46]. These sites, which we refer to as periphery websites, are defined by having a low global traffic rank. A better understanding of how users end up on these types of sites can aid in the development of protections for users. Beyond security, this information has applications for work related to spam [5, 60] and misinformation [42]. In this article, we identify the main pathways that lead users away from their routine web browsing and bring them to low-trafficked periphery websites. These pathways represent areas of opportunity for the design of future interventions to aid and protect users.

This study represents a substantial contribution in a series of web measurement studies conducted over the past three decades. With the most recent study having been run in 2011, our results help illustrate the evolution of web browsing over the subsequent 8 years. As summarized in Table 3, this study was conducted on a substantially larger scale, collecting more than 5.2M web requests across 257 participants over a period of 352 days on average. Of the previous studies, only one had more than 25 participants. In addition, the browsing data for this study was collected from Google Chrome users. In 2019, Google captured 61.8% of the browser market share as opposed to XMosaic (obsolete), Netscape (obsolete), and Firefox which maintained 34.1% of the market share at its peak in 2010 and only 4.8% in 2019 [55]

Our results indicate five primary changes in user browsing over the past three decades. First, the use of multiple tabs to switch between web pages has almost doubled as a proportion of all web navigation since 2011, rising from 28.6% to 54.4%. Since being introduced in the 1990s, tabbed browsing has grown in popularity among computer users by providing the ability to switch rapidly between tasks, open multiple links in the background, maintain frequently used pages, and provide short-term bookmarks [14, 23]. However, our findings represent a leap in adoption with tab navigation exceeding the number of web requests made by users for the first time.

Second, the use of the back button to open previously visited web pages has fallen from over 30% of web requests in the 1990s to 10.5% in the 2000s to 2.1% in our study. Previous work has suggested that back navigation has been replaced by the use of multiple tabs [14, 43]. Although this would fit the trends we observe across studies, ours included, we did not find evidence of an inverse relationship when examining the use of tabs and back navigation between users in our study.

Third, search engine functionality directly embedded in the browser has become a staple of user web browsing comprising 11.5% of all navigation. We find that the use of search terms in the address bar has eclipsed that of supplying URLs directly. The ability to provide search terms to the address bar was first introduced when Google Chrome was released in 2008 and was not available in Firefox 3.0, the browser used in the most recent 2011 browsing study. Within this category of navigation, we observe two distinct types of search behavior. In 99.9% of cases where search terms were provided to the address bar, the user submitted the query directly and was redirected to a search engine web page displaying the query results. In the remaining cases, the user selected one of the browser's cached suggestions that appeared as a set of drop-down options as they typed and was navigated directly to the destination website.

Fourth, the average revisitation rate, both conventional ($\mu \,=\,59.7\%, \sigma \,=\,8.3\%$) and effective ($\mu \,=\,78.1\%, \sigma \,=\,5.8\%$), is high. To interpret this rate, we make direct comparisons to the findings of Zhang and Zhao [63], who report both the conventional and effective rates. In both cases, we find a much higher rate of revisitation among users in our study. To compare to studies prior to 2011, we use the effective revisitation rate which accounts for the increased use of tabbed browsing observed in our study. By comparison, our revisitation rate is on the higher end of the spectrum, nearing that found by McKenzie and Cockburn [36]. Revisitation rates lack a mechanism to account for the length of time that data is collected. A revisit to a web page after 2 minutes is treated no differently than a revisit to that same web page after 2 years. Given the relatively short duration of previous studies, we speculated that the longer time period covered by the SBO might account for the increased revisitation. To test this hypothesis, we recalculated our effective revisitation rate limiting the data of all participants to the first 20 days of their participation, allowing us to roughly match the quantity of data collected by Zhang and Zhao [63]. However, our result using this method ($\mu \,=\,76.6\%, \sigma \,=\,6.7\%$) did not shift substantially. As a result, we conclude that users in our sample traveled back to sites they have previously visited at a higher rate than in most other studies.

Fifth, the number of pages visited per day has steadily increased over time. This is driven in part by changes user browsing patterns. In general, users are spending more time online as work, shopping, and entertainment become increasingly online activities. In addition, we find that users who switch between tabs more frequently visit more web pages. As we have already observed, switching between tabs as a form of web navigation has grown by 79% since 2011. Therefore, it follows that tabbed browsing behavior accounts for some of this growth. However, although behavioral factors do contribute to this trend, we hypothesize that technological changes are as much, or likely more, of a factor driving the increase in average web page visits. This includes rising broadband internet speed and the expansion of content delivery networks that deliver web pages to the browser faster.

Investigating the increase in the use of multiple browser tabs further, we find several behavioral trends associated with users switching between tabs. Figure 1 displays three graphs that illustrate how the use of tabs is associated with, from left to right, the rate of visiting web pages, the rate of visiting websites, and the average daily web activities. As mentioned previously, users who switch between tabs more frequently also view more web pages. As shown in Figure 1(a), a linear relationship (r = 0.97) exists between the rate that users switch tabs and the rate that they visit web pages. On average, users visit an additional web page for every 2.2 tab changes they make. This contradicts previous findings that showed greater tab switching reflects higher levels of multitasking but results in the same number of page views [23].

If we examine the number of websites a user visits, rather than pages, we find that two types of behavior emerge. As shown in Figure 1(b), we predominately find a linear trend that follows the leftmost figure: users who switch between tabs more frequently tend to visit more sites. Shown in blue, these multitasking users visit an additional website for every five to six tab changes made. In contrast, we find a smaller subset of users who exhibit linear browsing preferences. These users, highlighted in orange, visit websites sequentially while infrequently switching between tabs.

Finally, as illustrated in Figure 1(c), we observe a non-linear relationship between daily tab use and the average number of web activities that a user engages in. Here a distinct web activity corresponds to the user visiting a website of a specific category such as entertainment, social media, or communication. As such, users who visit websites across a broader set of categories per day are also said to engage in a greater number of web activities. Although the rate at which users switch tabs is weakly correlated with the number of activities ($r\,=\,0.35$), there is a much stronger correlation between the number of tabs used and web activities per day ($r\,=\,0.67$). This relationship exhibits a logarithmic form with large increases in web activity at low levels of tab use and smaller increases at high levels of tab use. This indicates that multitasking behavior does not necessarily mean that users engage in a greater number of web activities. However, the use of multiple tabs appears to facilitate users engaging in a greater number of activities throughout the day.

In addition to observing several behavioral trends related to tabbed browsing, we also find that the increased use of tabbed browsing behavior among users poses significant challenges for web and browsing research. As discussed previously, we observed that the use of multiple browser tabs to switch between pages has exceeded the number of traditional web requests for the first time. Since switching between tabs does not generate a web request on its own, less than half of a user's interactions with their browser are captured in server-side web logging or clickstream studies that focus exclusively on web requests. Although the sequence of links that bring a user from one web page to another remains unaffected, attempts to gain a deeper understanding of user behavior (e.g., analysis of how users gather information online) based on these methods are likely to be biased.

Furthermore, we find that the measurement of dwell time, an important metric in industry and research, is quite inaccurate if tabbed browsing is not accounted for. Dwell time refers to the period of time that a user spends on a given web page and is used in search engine optimization, advertising, and a variety of other applications. This is usually measured by computing the time between user web requests during the course of a browsing session. Typically, 10 to 30 minutes of inactivity is used to denote session completion [2, 8, 9, 19, 20, 26, 29, 32, 34, 46, 49, 56, 62]. We evaluated this method by applying it to a clickstream version of our own data and then contrasting the results, which are summarized in Table 4. Using a 20-minute threshold, we find that traditional clickstream models ignore nearly 20% of user dwell time. Furthermore, of the 80% that is captured, 37% of the time the user is on a different page than the one indicated by the clickstream model.

Accounting for tabbed browsing can remedy the dwell time accuracy issue and improve data coverage to 88.9%. The latter is done by extending the user browsing session beyond the last web request to the last tab navigation before 20 minutes of inactivity. The remaining 11.1% of coverage was obtained in our models by capturing mouse movements and the use of non-browser applications, eliminating the need to define a browsing session in the first place.

In line with the findings of Obendorf et al. [43], we do not find evidence that would support the concept of an average internet user. Within our sample of participants, we do not observe a normal distribution of user behavior or a single cluster of shared habits. Instead, we notice a broad spectrum of user behavior. For example, in Table 5, we find that users on average spend 2.6 hours per day on their computer. However, across users, we find a wide range of activity with one participant having spent less than 20 minutes per day on their computer and another who spent over 11 hours on average. A similarly broad spectrum of habits is observed across application use, browsing time, websites visited, page views, and web activities. We did observe one outlier whose high browsing use was a result of being in the study for a short duration (7 days) and having a browser hijacker installed on their machine. This adware multiplied the user's actual activity by redirecting web traffic and opening unwanted advertisements. We report average behavior across users both excluding and including, in parentheses, this user.

Although we did not expect to find a single average user, we had hypothesized that we would find several different web browsing behavior profiles based on the duration of user browsing, the frequency of multitasking, and the types of websites typically visited. However, the spectrum of browsing behaviors we observe does not lend itself to clustering users into discrete groups. The research team applied unsupervised clustering (DBSCAN) using several of the key metrics but did not find any distinct clusters. Furthermore, we did not find any strong relationship between these behaviors and different classifications of users based on their primary web activities. For example, user behavior was independent of whether the user primarily spent their time on social media websites as compared to those who mostly used their browser for work. As such, we echo the findings of previous work in concluding that there is no such thing as an “average user” [15, 43].

To provide initial context to user browsing behavior, we examine the role of the web browser in relation to other uses of the computer. Based on the collection of application data, a unique feature of the SBO, we observe that participants primarily use their computers to browse the internet. Figure 2 shows how users spend their time on the computer, grouped into categories. Dots represent individual measurements; the adjacent whisker plots provide information about the distribution of these measurements. As the figure shows, browsing serves as the primary function of nearly all of the participants’ computers. On average, browsing consumes 63% of the time users spend on their machine. However, we observe a wide range of use spanning from 8% to 95% of users’ time. Outside of browsing, users spend most of their time split between working using professional or educational software, configuring aspects of the computer or operating system, and directly interacting with the file system through actions like copying or moving files manually.

For most users, using entertainment applications outside the web browser does not consume a large portion of a participant's computer time. This is likely due to the traditional forms of entertainment, like movies and television, becoming increasingly available for streaming directly in the browser rather than requiring a separate media player. In general, this follows a growing trend in which the browser is becoming the centerpiece of the personal computer experience. Video games make up the majority of non-browser entertainment consuming 56.7% of users’ time in that category. Although some games have become browser based, many still require their own software applications to play.

Within the web browser, we find that users spread their browsing time across many different types of websites. Figure 3 illustrates how users spend their time on the web, grouped by website categories or “web activities.” On a daily basis, users engage in seven to eight web activities on average. Most commonly, users spend their time on work (16%), social media (15%), and entertainment websites (14%), with some users dedicating well over 50% of their browsing time to these activities. These popular activities align closely with the findings of An et al. [1], a study of the internet use of households in Belgium that was conducted at the network level.

However, web browsing is more than just how users spend their time on the internet. The way in which users navigate between different types of websites is equally important. Figure 4 combines these two dimensions to illustrate how users browse the internet. In this graph, categories of websites are represented by circular nodes and are color coded by type in the same manner as Figure 3. The 32 most visited websites, which account for 50% of user dwell time, are separated out and clustered together with the other websites of the same category. The size of each node represents the cumulative time users spent browsing those types of websites or website. The lines connecting various nodes indicate users navigating between different websites. The thicker the lines, the more users are navigating between the two types of sites. As such, this represents how frequently users are switching between different web activities. The color of the line indicates that users are primarily navigating away from websites of the matching color and landing on websites of another category. For example, the thick gray line connecting new tabs and Google conveys that more users open a new browser tab and then navigate to Google rather than vice versa.

Using this graph, we observe six common patterns in user browsing behavior. First, search engines (purple nodes) serve as the backbone of user navigation, allowing users to to traverse to other areas of the internet. This is exhibited by the thick purple lines, indicating outbound navigation, extending from Google to all other nodes in the graph. This indicates that search engines are heavily used to navigate to across the internet and switch between different web activities. In total, participants used search engines to navigate to more than 62,000 different websites, approximately half of all websites visited.

Second, a common navigational pattern among users is opening a new tab, switching to their search engine, and then navigating outward to other types of web activities. This is illustrated by the thick gray line connecting the new tab node and Google. We hypothesize that this represents a jumping-off point for user browsing and indicates the start of a new browsing task or sub-task.

Third, we observe several categories of websites that have primarily outbound relationships with other areas of the graph. This includes categories like entertainment, shopping, lifestyle, computers, news, reference, and finance. In these cases, users are primarily navigating to these sites from a search engine, spending time on that site, and then leaving for other areas of the internet. We believe that this represents when users are engaging in a specific one-off task like reading an article, watching a video, retrieving a certain piece of information, or handling a transaction.

Fourth, in contrast to the previous point, there are several categories that have primarily inbound relationships from other areas of the graph. Users are navigating to these sites from all different areas of the internet, not just from search engines. This includes social media, communication, and work-related websites. We hypothesize that these websites represent the more routine aspects of a user's web browsing. These sites are the ones most commonly used, those that are kept open in another tab that the user routinely switches back to, or ones that are bookmarked for regular access.

Fifth, we observe a core set of work-related activities and set of leisure activities along with a strong indicator that users multitask between the two frequently. On the right side of the graph, there are strong associations between work, business, email, and computer-related sites. Many websites in the computers category relate to programming and cloud computing. As such, we believe that these sites capture user productivity. On the left side of the graph, there are also strong associations between social media and entertainment activities. However, there are equally strong connections between the set of work and leisure activities as there are within these sets. This indicates that many users multitask, switching between work and leisure activities throughout the day rather than distinct blocks of time dedicated work and leisure individually.

Sixth, we find that large technology companies capture a substantial proportion of our users’ attention within specific categories. YouTube and Facebook share the majority of users’ time spent on social media, garnering 49% and 35% of total time, respectively. Google maintains a substantial proportion of user attention with Google Search accounting for 76% of search engine use and Gmail handling 54% of browsing time dedicated to communication (light blue nodes). Last, Amazon maintains a 40% share of users’ online shopping (dark gray nodes). We compare our findings, based on user dwell time, to publicly available estimates of market share, likely based on website visits, in Table 6.

Although we do not observe a large difference between metrics for Amazon, we do find a substantial difference for YouTube. This is a result of comparing metrics based on dwell time to that using visits. Users who go to YouTube, which is dedicated to video content, likely spend a much longer period of time on each page watching videos as compared to other social media sites that contain a greater amount of static content. Following this logic, we see skew in the opposite direction for Facebook. The overrepresentation observed for Google Search and Gmail are likely a result of collecting our data through the Chrome browser, another Google product. Given the interoperability of Google's tools, it is not surprising to find a bias toward Google-owned websites in our sample.

The dominance of these large technology companies is indicative of a larger trend of centralization to web browsing. Overall, most of our user's browsing time is dedicated to a very small number of popular sites. Of the websites visited by our participants, we observe that users collectively spend 30% of their browsing time on the top 4 websites, 50% of their time on the top 32 websites, and 80% of their time on the top 1,000 websites. Of the 123,646 different websites visited, this concentration of activity covers less than 1% of the total sites. This indicates that users habitually return to the same small set of websites over and over again where they spend the majority of their time. This finding aligns with the high revisitation rates we observe in our sample, further reinforcing that web browsing is both highly centralized and becoming more centralized over time.

The allocation of browsing time within our sample also closely aligns with global web traffic ranks. Figure 5 provides the cumulative distribution of browsing time spent across websites by global popularity, measured by the OpenPageRank initiative's traffic ranking. Overall, we observe a power law distribution in which user browsing time follows an 80/15/5 split. Users spend 80% of their time on core websites, the most popular sites that have a traffic rank between 1 and 1M. An additional 15% is spent on mid-tier websites, those with a traffic rank between 1M and 10M. Finally, periphery websites, which fall outside of the top 10M, capture the remaining 5% of user attention. This distribution of user activity across websites resembles a core-periphery structure that is often used in network theory. In this type of model, there exists a dense, interconnected core and a sparse, loosely connected periphery [3].

Within the top 10M ranked sites, there were several anomalous websites that garnered a disproportionate amount of browsing relative to their rank. Upon closer inspection, these sites were specific to the user's occupation, the university they attended, or their regional bank. As such, these spikes in the curve are not unexpected. Of particular interest is the accumulation of 5% of user browsing time on websites outside of the top 10M. The time users spend on these periphery sites cannot be explained in the same manner.

Previous work has demonstrated that websites with a lower traffic or page rank have a higher likelihood of containing risky content [24, 33, 35, 46]. Websites that experience high volumes of traffic are able to monetize through advertising, sales, or collecting data and have the resources to maintain the security of their site. Although most websites with low user traffic are completely legitimate, these sites have greater incentives to monetize through less reputable ad networks, contain unwanted trackers, or even contain malicious content. In addition, low-user traffic websites sometimes lack the resources to maintain proper security updates and can be hijacked by malicious actors.

In our data, we find that 89% of websites that were flagged by Google Safe Browsing as containing malware or phishing content were periphery websites that ranked outside of the top 10M sites. In one example, which is eerily reminiscent of earlier security work on web compromises [30], we found a periphery website run by a university research group that had been co-opted by external actors and transformed into a storefront for an illegal online pharmacy.

The left section of Table 7 shows the top 10 categories of websites visited on the periphery of the internet across all of our users. Over half of the visits to unranked periphery websites are related to video streaming, computers, and crowdworking. Many of the websites related to computers appear to be legitimate technical guides or small businesses. However, upon manually examining the websites in the streaming and crowdworking categories, we find more questionable content. Many of the streaming sites on the periphery include potentially pirated video sources. Furthermore, the majority of the crowdworking websites offer rewards and freebies in exchange for opinion surveys or personal information.

Table 7. Top 10 Website Categories That Users Visit Most Frequently on the Periphery of the Internet and the Top 10 Overrepresented Categories in the Periphery as Compared to the Distribution of Visits to Core Websites

Also displayed in Table 7 are the most overrepresented website categories that users visit on the periphery of the internet. This is reporting in terms of relative risk, otherwise referred to as a risk ratio. The risk ratio is calculated by taking the probability that a user visits a website of a given category when browsing periphery sites divided by the probability they land on a website of the same category when visiting core or mid-tier websites. This does not mean that more of these types of websites exist on the periphery of the internet. Rather, users are much more likely to visit these categories of websites when on the periphery. For example, users are 28 times more likely to land on an adware site when browsing the periphery of the internet than they are when visiting core or mid-tier sites. This relationship is descriptive rather than causal, and is indicative of the type of content that users are more likely to engage with when browsing periphery websites.

Of these categories, adware and streaming sites are particularly interesting as they both make up a substantial proportion of visits to periphery sites and are highly overrepresented. In addition, we find that users are much more likely to visit alternative health and science websites on the periphery. These types of websites, which often contain misinformation, have been shown to distort valid scientific information, perpetuate conspiracy theories, and negatively impact individual behavior and decision making [12, 16, 59]. This has important implications for researchers studying the dissemination of science-based information. Although much of the attention in this space has been directed toward social media [7, 39, 61], our results indicate that the scope of this problem extends well beyond these sites. In only 7.4% of these cases did the user navigate directly to an alternative health or science website from a social media site. This rises to 24% when accounting for indirect navigation where users visited a social media website before, but not immediately preceding, the site containing alternative information. Search (34.3% direct, 53.1% indirect) and crowdworking (15.5% direct, 31.0% indirect) websites were the other primary sources of visits to these sites along with a multitude of smaller contributing categories. We also find that users are more likely to engage with pornographic, gambling, and free streaming websites on the periphery of the internet. Last, in a manual inspection by the research team of websites pertaining to senior health organizations and personal pages, we find no unusual activity, just sites representing individuals, small groups, or businesses.

The higher relative risk of users interacting with adware, alternative information, and potentially illegal entertainment websites on the periphery of the internet makes this subset of the web of particular interest to researchers and practitioners. The sources of traffic that transport users to the periphery of the internet, which will be referred to as gateways, represent an important decision point in users’ browsing. At these points, users deviate from their browsing routines and traverse outward to a website that is not often visited. To identify these gateways, we assessed the series of websites leading up to the user landing on a low-visited periphery website. Specifically, these events were limited to a user's first visit to a periphery site and exclude any subsequent visits to the same website during the duration of the user's participation within the study. In total, we observed 61,135 instances in the data that fit this criteria. As illustrated in Figure 6, we find that users navigate through three primary gateways to reach the periphery: search engines, other periphery websites, and mid-tier intermediary websites.

The largest contributors of traffic to periphery websites are search engines, the traffic source for 26% of all new visits. As we have demonstrated previously, search engines serve as a springboard for users to navigate across the internet in general. As such, it is unsurprising that they are also used to land on new periphery sites. The second largest gateway to the periphery, contributing 25% of user traffic, is other low-visited periphery sites. This indicates that users tend to use search engines to jump from core to periphery sites, but once there they tend to traverse to other sites on the periphery. The third gateway to the periphery is mid-tier websites, with ranks between 1M and 10M. These sites contribute an additional 17% of traffic to the periphery. More importantly, we observe that these sites serve an intermediary role in user navigation. Users tend to link from a core website to a mid-tier website, spend some time on that site before using it as a stepping stone to travel further out to a periphery site. In total, these three patterns of navigation account for more than two-thirds of all user visits to new periphery websites.

In addition to the primary gateways, we find that users link from streaming, adware, and adult websites less frequently, but at a disproportionate rate relative to the frequency of users visiting those websites. This bears close resemblance to the types of websites that are overrepresented in the distribution of periphery sites. Interestingly, 29.7% of visits to new free streaming sites originate from core entertainment websites, an indicator that users might first search through normal channels for the content they want before resorting to other free streaming options if not available. Of the other main sources of traffic, we observe a substantial proportion of the visits that come from crowdworking websites are directed to small opinion survey, user testing, and rewards websites which are often a part of crowdworking tasks. Traffic from social media sites lead to one of the most diverse sets of websites by type, second only to search. Users who navigate from retail websites often land on other smaller shopping websites or business homepages. However, it is unclear if these users are intentionally navigating between these specific sites, such as to compare products, or if they clicked on an advertisement that led them there.

The results of our study are limited in several ways. First, our data is based on the computer and browsing behavior of users on desktop and laptop computers. Given the predominate use of individual apps rather than a browser on tablets and smartphones, browsing behavior on mobile devices is likely very different. Second, our sample is based in the United States and primarily located in one major metropolitan area. Although there were several users who regularly visited international websites or those in different languages, the overall geographic and cultural composition of our sample limits the scope of browsing behavior that we observe. Users in other countries are likely to visit an overlapping, but largely different, set of websites on a regular basis. These differences likely hold across cultural differences within the United States, albeit to a lesser extent. As such, we are likely to find different gateways being used to access periphery websites across disparate populations. Third, our data was collected from users who owned computers running Microsoft Windows operating systems and whose browsing was done using Google Chrome. Fourth, despite the SBO data covering a substantially longer period of time than previous studies, our findings still represent a temporal snapshot of user behavior. As we have demonstrated in comparing to previous work, user browsing behavior has changed over time. With the continued shift in the population of web users, the trend of moving applications directly into the browser, and the emergence of new browsing technology, it is inevitable that user behavior will continue to evolve.

Although we observe several trends in user behavior and substantial changes over time, like previous work we find no clear definition of an average user. In the development of websites and web applications, creators often design and test products for specific types of users. Related research also falls into the same pattern when testing for security and usability across discrete groups of subjects. Based on our findings, this characterization can be problematic, leading to software built and tested well for some users and not for others. Instead, we recommend that developers and researchers treat internet users as a spectrum rather than a type.

This recommendation has an important implication for the use of a browsing session in future web research. These sessions, which are extensively used to artificially segment a user's web browsing for analysis, are often separated based on a fixed period of user inactivity. However, given that we observe a broad range of browsing habits across users, a single definition of a browsing session may be not be appropriate. Future research is needed to assess browsing sessions as a concept, evaluate whether a different paradigm is required, and determine the optimal unit of measure for analyzing user web browsing.

Until techniques are fully developed to address the noise in server-side measurements stemming from tabbed browsing and automatically generated web requests [54], future research on user browsing behavior should focus on client-side data collection. Conclusions regarding browsing behavior should be drawn from server-side measurements cautiously. However, we acknowledge that client-side measurement studies do pose their own significant challenges. Although server-side measurements can be scaled to thousands of users, such efforts using client-side collection methods are more expensive and technically difficult to manage. As such, future research that helps to mitigate server-side measurement errors would be immensely valuable to the field.

Based on our experience running this study and the limitations of current server-side collection methods, we recommend that studies of user web browsing satisfy the following six conditions. First, collection should be conducted on the user's own machine to obtain accurate measurements and maintain ecological validity. Browser extensions were particularly useful in deploying sensors on the user's computer while minimizing the technical challenges often associated with client-side measurements. Second, the platform for data collection should be lightweight enough so as to prevent interference with the user's normal browsing. Third, web requests initiated by the user must be distinguishable from those generated automatically by web pages. Although Vassio et al. [54] demonstrate a viable method of discerning user web requests from HTTP traffic, this option is not preferable to differentiating requests on the client side if available. The growth of HTTPS traffic, the constant evolution of the internet, and the changes in user behavior over time make this classification task very difficult and will inevitably introduce noise. Fourth, the use of tabs should be captured, especially the events where a user switches the tab that is currently in view. Fifth, events when the web browser application starts, stops, and moves in and out of the foreground should be recorded to bound user browsing time. Sixth, ideally some form of user interaction such as mouse movements, keystrokes, or touch should be collected to determine when the user is actively browsing or has just left their browser open on the screen.

The periphery of the internet poses important questions and challenges across several areas of future research. From a security perspective, the greater likelihood of users encountering adware sites on the periphery supports previous findings associating malicious content and low-traffic websites [24, 33, 35, 46]. To better inform and protect users, future research should investigate possible interventions that could be deployed along the common pathways leading to periphery websites. In addition, the substantial overrepresentation of websites related to alternative science and alternative health on the periphery of the internet has important implications for the dissemination of science. Future research in this area will help illuminate the situations in which users encounter this kind of alternative information for the first time. Such insights can provide researchers with a better understanding of how users evaluate information from different sources and how their consumption of information may change after being exposed to different ideas. Our findings indicate that users get to websites containing alternative information from many sources, and social media, although a substantial contributor, is not the primary source. The relationship between other contributing sites, particularly search engines, and alternative information should be studied in greater detail. Last, we observe a large proportion of visits to free streaming websites on the periphery of the internet. This is a negative sign for the entertainment industry and piracy researchers. However, with approximately 30% of visits originating from core entertainment websites first, at least some users appear to be resorting to free streaming sites rather than actively seeking them out to begin with. In these cases, it may be that the show the user is interested in is unavailable or too expensive through legitimate channels. Further investigation into user decision making is warranted.

Finally, two growing trends in web browsing demand further investigation. First, the increasing use of smartphone and tablet devices to browse the internet represents an important set of research that runs parallel to our own. Early mobile browsing studies have begun to draw comparisons and identify differences between mobile and personal computer browsing [11, 50]. However, changes in technology, especially in the area of mobile devices, have rapidly advanced in the past 8 years. In addition to the internet evolving over this period, mobile devices have become much more powerful and mobile internet speeds have greatly increased. Similar to our findings for desktop users, these developments likely correspond to changes in user browsing. As such, an update on the state of mobile browsing behavior is critically needed.

Second, applications are moving directly into the browser in increasingly greater numbers. This trend has been led in part by collaboration platforms like Google Docs and Microsoft Office Online. In fact, this article was written almost exclusively within a web browser. The recent development of technology like WebAssembly, which enables an even broader range of applications, such as arcade video game emulators, to run in the browser as they would on “native” hardware, points to the continuation of this trend. In the near future, we might find that the browser consumes what remains of users’ non-browser activity. As the browser continues to become the focal point of personal computer use, issues of security, privacy, and usability grow. Addressing these challenges relies on a foundation of knowledge about how users browse the web. Although our work contributes to this effort, it only represents a step in the larger longitudinal study of web browsing. Future contributions to our understanding of user browsing behavior will be needed.