How we test VPNs?

Network and security protocols need to be in harmony, so when testing a Virtual Private Network (VPN), you're looking at a multifaceted system that consists of software.

How we test VPNs?

Regarding a VPN, claims of being the "fastest" or "most secure" are not to be taken at face value. Here at PCRisk, we know that our readers count on VPNs to protect their online privacy, and so we conduct rigorous testing to give them a clear understanding of what they're getting. Comprehensive testing is key to spotting serious issues, such as data leaks and subpar encryption, which a poorly tested VPN review could miss.

Introduction

We're not just checking boxes, we're putting them to the test in real-world conditions when we test VPNs. Knowing how serious a misstep can be, we set the scene for the importance of a thorough VPN test. Where a poor choice can mean compromised data and subpar performance. Established VPNs are subjected to a very systematic and meticulous testing procedure to ensure that any VPN we recommend meets the highest standards for privacy, security, and ease of use.

Why You Should Trust PCRisk's Expertise

PCRisk.com relies on its years of experience in cybersecurity, comprised of experts and researchers who've studied the intricacies of malware and protective technologies, when testing VPNs. We bring a security-first mindset, we're not just average users, but people who comprehend the technicalities, encryption methods, and networking principles that underpin a VPN's efficacy.

We at PCRisk use the same analytical intensity and impartiality in our VPN assessments that we apply to our malware analysis. We go through a rigid process, ensuring that each VPN is evaluated equally. We don't let money or preconceptions sway our reviews, they're completely unbiased. Because we're not satisfied with just the theory, we verify every claim in real-world, we don't count on marketing or specifications to tell the story, we go by real results. 

We check speed, security, and streaming functionality, and our verdicts are based on tests we ran. Well-known for our unflinching commitment to accuracy, our findings in VPN reviews are backed up by the collective knowledge of the team. We're not afraid to be frank about it all, and we're driven by a genuine need to inform and protect our readers.

How We Test VPN Security

For security, a VPN's performance is put to the test here. We delve into the encryption the VPN uses, and expect anything modern to use AES-256 or the equivalent ChaCha20 cipher, along with secure tunneling protocols like OpenVPN, WireGuard, or IKEv2/IPSec. We check to see if authentication and handshake methods follow best practices. Such as using SHA-256 hashing or stronger, and RSA keys of 2048-bit or higher, plus we verify that it's implementing Perfect Forward Secrecy so that even if one encryption key falls into the wrong hands, it won't be able to be used to snoop on past communications.

We also thoroughly test for IP, DNS, or WebRTC leaks, connecting to the VPN and then running diagnostic tools and websites to see if our real IP address is ever exposed outside the encrypted tunnel.

A VPN that does so fails its primary purpose, so we're extremely serious about this test. We also simulate network dropouts to see if the kill switch works. A kill switch should automatically block all internet traffic if the VPN drops out. If a single packet slips out when this happens, that's a major red flag. VPNs that consistently prevent data from leaking out and keep their encryption integrity when things get rough pass our security review.

Beyond encryption and leak prevention, we also examine the VPN's infrastructure, favoring providers that operate their own DNS servers and offer advanced security features such as RAM-only servers, which wipe all data upon reboot, and colocated servers under their full control. Our rigorous testing confirms that these security measures remain effective under real-world conditions, and only when a VPN meets our stringent requirements do we declare it safe to use.

Evaluating Speed and Performance

The encryption and routing that VPNs use inevitably cause some slowdown. But the best services will do their best to keep this to a minimum. PCRisk tests each VPN, first measuring the baseline internet speed (without a VPN) in our network, which is what we compare the VPN's performance to. We usually hook up to servers in different parts of the world for each VPN, typically a local server (for example, within the same country or a country in proximity), a mid-distance server (perhaps in another continent, like Europe to North America) and a far-flung one (such as across the world in Asia or Australia), and check the speed at different times of day. This includes morning, afternoon, and night to observe performance under various network conditions and peak usage periods.

Our test logs the download speed, upload speed, and ping (latency) rates for each test run, using tools such as Ookla's Speedtest or similar, and repeating the test multiple times to verify the results. By taking an average of these results, we can get a realistic idea of what a VPN can deliver. We usually average the results of nine data points. Three servers get tested at three different times to get a final speed score. In our lab, we use a lightning-fast wired connection that can reach gigabit speeds or above, so we can eliminate the possibility of any slowdowns caused by our network. This rigorous approach allows for fair comparisons between VPNs under the same conditions.

Numbers alone don't tell the whole story, so we put VPNs to the test in the real world too: Streaming high-definition and 4K videos to check for buffering, downloading large files and torrenting (where possible) to see how they perform over time, and even running online gaming tests to measure latency and stability. We're watching out for how quickly VPNs hook up and stay connected, and if they can handle multiple protocols, we'll test the speed of each one. WireGuard vs OpenVPN is often used to see which gives the best result. Our goal is to let our readers know what they can expect from a VPN and whether it can handle all the data-intensive tasks they throw at it. Consistency in speed across various servers and times is what sets our top-rated VPNs apart.

Testing Streaming and Geo-Unblocking

PCRisk tests its ability to access geo-restricted content, essentially streaming services, websites, and platforms that are not available in our area, and this is given a lot of attention when reviewing a VPN. We check how well each VPN works with primary streaming services that people care about. Netflix, Amazon Prime Video, Disney+, Hulu, and regional services such as BBC iPlayer, HBO Max, etc. We use the VPN to try and stream content that's normally unavailable to us, by plugging into the VPN's servers in the country where the content is located (for example, using a US server to watch the American Netflix library from abroad).

We see if the VPN successfully opens the service and whether playback is smooth. Does the video kick in with high-quality and doesn't take forever to load? Can we sit back and watch for hours without problems or error messages? If the VPN gets blocked by a service (for instance, if we get the dreaded "You seem to be using a proxy/VPN" message), we'll try a different server or location to see if any combination can get around the restriction. We go beyond just checking them on web browsers. We test their streaming capabilities by checking how well they work on different streaming apps on various devices, because some services use VPN detection that's different from their websites. With our all-encompassing approach, we can tell just how reliable a VPN is regarding streaming media.

We also assess general geo-unblocking by visiting region-locked sites and services. We test if the VPN can access social media, VoIP services, or anything else that's restricted in certain countries, and if it works in heavily censored regions. If a VPN claims to function in countries with very strong censorship such as China, Iran or others, we look out for specialized modes or servers, known as stealth or obfuscated servers, that they may have to evade firewalls, and we take note of the special settings you need to set up to get these features to work. Our streaming and geo-access tests provide a clear picture of what online content you can access with the VPN. Top-tier VPNs get into many different streaming services and maintain excellent quality, and more mediocre VPNs may stumble or require jumping from one server to another. We make sure to highlight these differences in our reviews, since streaming access is a key use case for many VPN users.

Privacy and Logging Policy Analysis

When assessing a VPN service, we check that their logging practices and privacy policies live up to the VPN's purpose of protecting our users' privacy. Our in-depth evaluation begins with a meticulous analysis of a VPN's privacy policy, terms of service, and any other statements on data collection. We're looking for a strict no-logs policy, and for a VPN that says it's "no-log" or "zero-log", we verify what that really means. In most cases, we send them questions or check for independent documentation to confirm that they don't retain any data that could be used to identify a specific user, including source IP addresses, websites visited, download activities, and DNS queries.

Some VPN services have expressed that it is impossible not to log, but, as VPN users, we do not settle for "we cannot". We call out companies if detailed logs of user activities appear. Some privacy-respecting VPNs can indeed have very minimal information being used for purposes of network optimization and maintenance, for instance, recording the total volume of bandwidth used, or the exact time that a connection is established. Any user identification information that it gives out, we strongly object to, including activity logs. If there are no independent security or privacy audits, we look elsewhere. And look for any available audit reports, giving us that additional peace of mind. When rating VPNs, we check to see if they're hiring third-party auditors to verify their no-logging claims.

Coming from the company's word alone isn't enough, so we also look into the company's background and jurisdiction. A VPN based in a country with strong privacy laws and isn't part of any international intelligence-sharing alliances, like the "Five Eyes" or "14 Eyes" groups, is a good sign. Countries known for heavy data retention and government surveillance aren't ideal. If a VPN has a history of misrepresenting their logging policies or a data breach, we take that into account. Here at PCRisk, we aim to give you a well-rounded perspective of a VPN's treatment of your data and privacy, and only recommend services that show a genuine commitment to those principles. If a VPN can't be trusted to keep no logs, we'll let that reflect in our rating, even if the service is technically sound.

Usability and Features

Even if a VPN offers top-notch security and speed, its usefulness is severely diminished if it's a pain to navigate. At PCRisk, we take usability and features to be just as important as part of our evaluation. We look at how simple it is to sign up, subscribe, and install the VPN on various devices. We check out its dedicated apps on Windows, macOS for desktops, and Android, iOS for phones. If the VPN happens to offer browser extensions, we test those as well.

We look for accepted principles of user experience, such as a user-friendly interface and logical controls, in the VPNs we test. We expect that picking a server, establishing a connection, and changing settings should be intuitive, and that the options are clearly explained. Messy and disorganized interfaces can be daunting, especially to those who aren't computer-savvy, and we make sure that our tested VPNs are crystal clear. 

We also pay close attention to the stability and general usability of the app. We note any glitches, crashes, or connectivity issues and check how quickly the VPN app connects and whether switching servers, setting custom settings, or enabling kill switches is easy. We test features like auto-connect, which automatically kicks in on startup and when you move onto untrusted networks, and how the app can navigate network changes, like switching from Wi-Fi to mobile data. All these nitty-gritty considerations feed into our usability score.

Feature sets, finally, are another thing we check out, for they vary greatly from one VPN to another. We have a look at its more advanced features when we're testing a VPN. Split tunneling lets you route particular apps or services outside the VPN tunnel. We check that this works by, for example, excluding a single app and confirming its traffic goes through the regular connection. Multi-hop or Double VPN is a feature that sends your connection through two VPN servers to amp up security, and we check the performance and stability of this feature, too.

We also see if there are any built-in ad blockers, onion over VPN (Tor over VPN) support, or P2P-optimized servers for torrenting, and if a VPN has specialized servers for streaming or gaming, we fire them up to see how much of a difference they make. We also note the number of simultaneous connections the VPN allows, since this affects how you can use the service across all your household devices.

Customer support, we believe, is a massive part of the user experience, so when reviewing a VPN, we send a few standard questions to their support desk, usually via live chat or email. We see how quickly they respond, and whether they can provide answers to our questions, or if they send generic ones. Availability of 24/7 live chat, comprehensive FAQs, and clear setup guides, we're also taking into account in our evaluation. A VPN that's intuitive, feature-packed, and backed up by lightning-fast, knowledgeable support will earn top marks in this area. In contrast, clunky apps, omissions of necessary features, and subpar support knock our rating down. In summary, we test VPNs in a way that mirrors a real user's experience - from installation to everyday use - to ensure that the service is secure, convenient, and capable of meeting various needs.

What Hardware and Network Setup We Use

PCRisk sets up its tests in a controlled laboratory environment using the best available hardware. When evaluating VPNs, we want the results to show how well the VPN is working, not any limitations of our own equipment. Our test rigs go beyond the minimum requirements for VPN software, like a modern Windows 11 PC with lightning-fast processors that effortlessly handle the encryption/decryption tasks, and lots of RAM to prevent the client side of the VPN from slowing things down. Coming from the Apple side of the fence, we test on macOS systems and also use newer Android and iPhone phones, so we know how the VPN performs on different hardware and operating systems.

Our systems are regularly updated with the latest OS patches and drivers, and we usually run the test on a brand-new system or a clean virtual machine so that we don't get any funny results from old background programs or configurations. Virtual machines are also handy for some of the tests we need to run because they can isolate the VPN connection and simulate particular operating system versions.

We hook all our test machines up to a stable, high-speed internet connection, a fibre optic connection with bandwidth of 1 Gbps or higher, ensuring the speed measurements we get are coming from the VPN server, not our own internet. We understand that the maximum speed of the VPN and the speed of our testing line, 1000 Mbps, will tell us if the VPN is the limiting factor when testing a VPN. When testing, we try to eliminate all other heavy network activity, so that we're working with a controlled environment. Sometimes, we test the VPN over a standard home Wi-Fi network (with a more typical bandwidth like 100 Mbps) to see how it will perform in a more realistic scenario, but our main tests are done over our high-speed line to eliminate any faults.

Noted leak test techniques include using a local DNS server and IP/DNS logging tools within our network to catch any errant packets that might slip through the VPN tunnel, and our custom lab allows us to simulate all sorts of network conditions. We can add packet loss, increase latency, and test other types of network failures to see how the VPN protocol responds. We can also test how well the VPN app works with different types of networks. Wi-Fi, wired, and even mobile tethering are able to standardize the setup of every VPN, so that the competition all gets put to the same test.

We use a local DNS server or IP/DNS logging tools to capture any stray requests that may leak out of the VPN tunnel. Our lab environment also allows us to simulate various network conditions, such as packet loss and changed latency, and test how the VPN protocol responds to these conditions. We can further test the VPN's transition speed when shifting between different types of networks, such as Wi-Fi, wired, and mobile tethering, and can standardize our testing setup so that every VPN is tested to the same level. This makes for a fair and apples-to-apples comparison that gives our readers, and you, reliable results.

How We Score VPN Services

When evaluating a VPN, we gather all of our test data and observations and feed them into our algorithm. Our scoring system is constructed to be straightforward and considers what matters most to the average user. Here's how we break it down:

Security (Encryption & Leak Protection)Weight: Highest. Security is our top priority when examining a VPN, and this area of our review has a huge impact on the final score. We check how well a VPN encrypts our data, the strength of its kill switch, and whether it leaks any sensitive information, and a VPN that gets all of these things right will be awarded a top score, heavily contributing to the final rating. If a VPN fails in any of these areas,  the result is a very low rating.

Privacy (Logging Policy & Trust) - Weight: High. When rating a VPN, the level of commitment to user privacy is right up there with technical security. We use a three-pronged approach to evaluate a VPN's logging policy, jurisdiction, and transparency, and basically look for no-logs policies, VPNs that have a good track record of privacy, and don't have a history of misusing user data. We deduct points if we notice any logging or anything else that doesn't feel right. After all, even a technically secure VPN isn't worth recommending if the company behind it doesn't respect user privacy.

Speed and Performance - Weight: High. We aggregate the results of our speed tests and real-world performance trials into a performance score. VPNs that consistently deliver fast download and upload speeds and low latency will score well. We also factor in stability (no random disconnections) in this category. Speed is important for everyday usability - a VPN that drags your Internet speed down significantly will lose points. This category has a high weight because performance impacts user satisfaction directly, whether for streaming, browsing, or video calls.

Streaming & Geo-Unblocking - Weight: Moderate. With respect to streaming, not everyone uses a VPN, but it's a popular use case, and the services that can deliver in this area are given a higher rating. Successfully unblocking multiple popular platforms and maintaining HD/4K streams without issues will earn a service a strong score in this area. While security and base speed are priorities for all users, a VPN's ability to stream doesn't have to be a basic function. We've given this aspect a moderate weight in the final score. If a VPN fails to access one of the major streaming platforms, it will still have its uses, but won't be as versatile.

Usability & Features - Weight: Moderate. The user experience matters most, including how intuitive the app is, the breadth of features, and the reliability of the VPN. We combine the usability of the VPN, the reliability, and the utility of features such as split tunneling, multi-hop, etc. To come up with our score for this category. A VPN that is simple yet flexible and doesn't mind sacrificing some usability for bonus features will do well, but has to still be able to make up for this with very high security and very fast speeds. We also include our assessment of customer support here. A provider that backs its product with great support and guides may get a small boost. This category has a moderate weight: not as high as core security or speed, but enough to distinguish polished services from clunky ones.

Each of the above areas is scored individually during our review process. We then calculate an overall score for the VPN, weighted according to those priorities. For example, security and privacy combined might make up nearly half of the total points, reflecting their importance. Speed and performance might contribute a significant portion as well, with streaming and usability/features rounding out the rest. We believe this weighted scoring gives a fair overall picture. A VPN can't shoot to the top of our rankings with fast speeds alone if its security is subpar. Conversely, a highly secure VPN must also be reasonably fast and user-friendly to earn a recommendation.

Finally, we provide a transparent explanation of our scores in each review. We want readers to know why a VPN earned its score. If one category pulled the score down (for instance, great security but weak streaming support), we'll clearly mention that trade-off. Our scoring system is there to guide users by summarizing a lot of complex test results into an easy-to-understand rating. However, we always encourage reading the full review details - which our "How We Test VPNs" article here has hopefully illuminated - so you understand the nuances behind the numbers. PCRisk is committed to fairness and accuracy in our scoring, and we periodically revisit and revise scores if services improve or standards evolve. Ultimately, our goal is to ensure that our readers can trust these scores as a reflection of real-world VPN performance and reliability, based on a rigorous and impartial testing regime.