How We Test VPNs

    
    Here’s how we test VPNs at CNET.
    Sarah Tew/CNET
    

    A virtual private network allows you to create a more private connection between your computer and the internet over any network — whether that’s public Wi-Fi at an airport, your mobile carrier’s network or your own home network — by creating an encrypted tunnel through which your browsing information is sent. While VPNs can’t make you entirely anonymous, they are a cornerstone technology for improving your digital privacy. They can protect your browsing from intrusive data collection of your internet service provider, help shield you from many forms of mass domestic surveillance, and can particularly improve your security against malicious actors while logging into online accounts in public.?
    As the use of commercial VPNs rises sharply, and competition between companies grows fierce, the need for critical, objective evaluation has become paramount. We’re constantly improving our VPN review process with the help of your feedback, new testing methods and advances in technology. Here’s how we review them now.?
    Editors’ note, Feb. 9, 2022: The VPN industry has undergone significant change in the past few months, with?all three of our top VPN choices announcing major changes?in corporate ownership. In December,?ExpressVPN announced that it had officially joined Kape Technologies, a company that already owns several other VPNs and has?raised privacy concerns in the past. In February,?NordVPN and Surfshark announced the two companies were merging, though they’ll continue to operate autonomously. We’re in the process of reevaluating all of our top picks in light of these changes. We will update our reviews and, if necessary, our rankings to account for this new competitive landscape.?
    Read also:?Best VPN service of 2022
    How we evaluate speed
    Any use of a VPN, no matter how fast, will somewhat reduce your browsing and loading speeds. Speed alone won’t earn a VPN our recommendation, but for many users, speed is the most important factor when choosing a service. Our approach to evaluating speed includes a number of critical questions.
    Does the VPN intentionally throttle your speed? Does it limit the number of times you can switch servers? Does it cap your data or usage? All of those practices can slow you down, so a VPN’s answer to all of those questions should be “no.”
    A wide, well-maintained network of servers enables faster speeds, so we look at how many servers each company has, where they are located, and what kind of hardware is used. We also note how many IP addresses a VPN offers, and whether they offer dedicated IP addresses.?
    Speed tests are currently conducted manually using OpenVPN protocol — generally considered the most secure, and most widely used type of open-source protocol. Once we test our internet speed without a VPN, we connect our machines to the VPN, and pick five servers in diverse locations. We test those five servers, five times each, at intervals over three days via the widely used Ookla Speedtest. Then we average the scores. We also offer, when applicable, subjective appraisals of the relative speed with which a VPN client is able to make a connection, or to reconnect after we close the application.
    Read more: Fastest VPNs of 2022
    We recognize that the development of proprietary protocols among individual VPN providers has continually increased in recent years, particularly those proprietary protocols?built on WireGuard. Unique protocols may provide each VPN with its own distinct speed advantage compared to the speeds it may achieve with OpenVPN protocol, and we will note where applicable any standout differences we observe between the comparative improvements or decline of speeds based on protocol changes. However, our baseline speed evaluations will continue to be based on those achieved using OpenVPN options, to ensure readers have an idea of what speeds to expect from a VPN while using the protocol we most recommend for safety.?
    While we maintain a focus on controlling a reasonable amount of variables to provide consistent and fair evaluations, our overall goal in speed testing is to present readers with a realistic expectation of average commercial VPN speeds by replicating some of the conditions in which they are likely to use a VPN service. Readers are unlikely to use a commercial VPN in laboratory-like conditions with business-class internet access and the fastest possible devices. Instead, most people are more likely to use a commercial VPN at home or in transit, with average residential speeds that may be shared among other networked devices, with mobile speeds that may or may not achieve 5G and vary with travel, and with devices that may not be the newest or most powerful on the market.?
    Even as we continue to refine our speed testing methods, we find our most useful and practicable advice to the average VPN user is uncovered while actively replicating the garden-variety speed bumps common to the widest swathe of connected users.
    Read also: Your VPN won’t save you from these 3 things
     How we evaluate security
    We closely scrutinize the security background and claims of VPN companies. Without security and privacy protections, a VPN loses the ability to perform its core function: to help you access and share media across the world.?
    First, can the VPN be used safely inside of countries with censored or firewalled internet, such as China? For casual users in the US who simply want to safely browse on public Wi-Fi, a VPN’s availability in a heavily censored country may not be paramount to a purchasing decision. But for an increasing share of the world, that availability is the single most important factor in choosing a VPN.
    Second, does the VPN allow secure torrenting and other peer-to-peer connective sharing? Some VPNs actively discourage users from this technology.?
    Finally, can you access Netflix and other media-streaming sites with the VPN? You should be able to. Accessing geoblocked content is key to using your subscription services while traveling.?
    Because seemingly small bits of your data can be used to build a unique profile with potentially identifying information, we seek to identify any weak points in a VPN where your data may get through. We test for privacy issues such as DNS leaks, IP leaks and WebRTC leaks — all of which are security flaws that could share your data with outside parties — using publicly available tools like the DNS Leak Test from Perfect Privacy, IPLeak and IPv6 Test.?
    We look for VPNs that offer security tools like Perfect Forward Secrecy, split tunneling, aggressive ad blockers and any unique security features.?
    Jurisdiction and ownership transparency are also important considerations to us. While user needs may vary, the ideal VPN would be located outside of those countries which participate in US intelligence-sharing agreements, with an ownership structure visible for public verification. Likewise, we look at every VPN provider’s privacy and data collection policies, along with those of any parent or sibling company where applicable and possible, to discern for potential privacy blind-spots.
    Because traditional commercial VPNs all require user trust to some extent, we look for as many efforts at transparency as possible by a provider. These generally include, but aren’t limited to: the use of bug bounty programs or similar methods to invite continual improvement, the timely public disclosure of any potential privacy or security threat, the publication of third-party security audit results from auditors without apparent potential for conflict of interest, the open-source publication of any central component’s code, continued emphasis on the functionality of OpenVPN protocol wherever feasible, and the exclusive use of RAM-disk servers (except in places where virtual servers may be the only functionally safe user option).?
    Read also: How to set up a VPN on your iPhone or Android phone: Yes, you need one
    

    
    
VPN security and privacy deal-breakers
    We specifically single out three security and privacy factors that we consider deal-breakers.?
    If a VPN is caught keeping or sharing user activity logs, we will not recommend it. While most VPN services claim they don’t track or keep logs of user activity, those claims can sometimes be impossible to verify. We do our best to vet VPNs for prior incidents before recommending them, and our rankings prioritize VPNs that have undergone — and published the results of — an independent third-party audit of their operations. If a VPN service has been caught keeping logs, we also look at that VPN’s parent company; if its parent company owns other VPN services, those sibling VPN services may also be advised against after evaluating any potential privacy risks.?
    Another deal-breaker for us is when a VPN doesn’t uphold minimum encryption standards. Users should expect AES-256 encryption or better from VPN services.?
    Finally, if a VPN doesn’t offer a properly functioning kill switch, that’s a deal-breaker. When a VPN connection is suddenly interrupted, a kill switch feature may be the only tool to prevent a sudden exposure of your device’s data and traffic.?
    Read also:?All the VPN terms you need to know
    How we evaluate VPN expense
    We test VPNs using review subscriptions when available, or by otherwise purchasing the service directly. When you click through from our site to a VPN service and buy a product, CNET may earn affiliate commissions. Our reviewers do not. Nor do our reviewers accept payment or incentive in any form from any VPN service they review. Our reviewers critically scrutinize VPN services with complete editorial independence.?
    We compare a VPN’s price to its competitors’. We look for VPNs that offer more flexible payment options for those customers who prefer to use Bitcoin or PayPal, and for those customers who prefer month-to-month payments instead of annual lump-sum payments.
    We verify the number of simultaneous connections you can have on a single subscription. We install the software on multiple devices — running MacOS, Windows, iOS and Android — not just to check compatibility and functionality, but to evaluate the usability of the software for different users.?
    The devices may vary, but we aim to test VPNs with the latest versions of as many as we can. And on each device we only test after wiping the device and working from a freshly installed operating system. We aim to expand testing to additional devices like iPads and the Nintendo Switch as well.
    We expect great customer service to be available 24/7 via chat, and prefer those companies that also offer phone support where possible. We prefer a minimum seven-day trial period, but expect a 30-day money-back guarantee at a minimum.
    Update, Sept. 22, 2021: Our speed evaluation section has been updated to reflect the needs of testing emerging proprietary protocols offered by VPN providers, and to include a clarifying emphasis on how we shape our testing methods to replicate the experiences of the average VPN user. Our security evaluation section now includes clearer articulation of vetting practices we’ve long had in place. First, we’ve noted our preference for companies that maintain open-source protocol options and bug bounty programs, and which have undergone third-party audits and server network improvements. Secondly, we’ve also more directly described how we evaluate those VPN providers whose sibling companies have been caught keeping user logs.?