Versus Pauses Operations to Audit Security

Versus Market, a “security driven marketplace,” entered maintenance to conduct a security audit in response to a recent concern involving an IP address leak. The leak, according to the market, is not an issue for Versus users.

Updated on March 24 to add a statement from Versus.

Yesterday, Versus staff /u/WilliamGibson posted an announcement in the Versus subdread concerning the “potential IP leak” as well as the related downtime:

We received reports of a potential IP leak from one of our middleware server and decided to shut the market down temporarily to investigate the issue before any damage is done.

The affected server has been added recently during the ddos and is not a backend server. It has been wiped and abandoned for security reasons.

I don’t want to cause any panic and can assure you that I only post this for transparency.

The market will be running as usual after we made sure there are no other leaks in max 12 hours until 11:00 UTC.

Please excuse the inconvenience caused and rest assured no userdata or any critical information in general has been leaked.

An individual operating under the username “BenedictParchezzi” is responsible for discovering the IP address of one of Versus Market’s middleware servers. BenedictParchezzi made the discovery after targeting the market with a DDoS attack. According to the attacker, Versus handled the disclosure “honestly.”

If the announcement from Versus staff is accurate, the issue is relatively minor. An administrator of a different marketplace described the incident as “not good, but not that bad either.”

Update from /u/WilliamGibson:

What happened?

A few days ago a DDOS against Versus started.

The application layer of Versus is very strong, so it was tor failing sometimes.

The attacker found an application layer vulnerability by requesting non existing captcha images. This lead to a lot of useless file IO and exhausted the main server resources.

Changing the captcha system isn’t a quick fix, so we deployed a custom middle-ware on the tor fronted servers to deliver the captchas and filter bad traffic before it even gets to the main server.

To further improve the resistance of Versus we did a V3 load balancing test.

To not have to apply the middle-ware to all servers for this test, it was moved to a single server in the same DC the tor front-end servers.

The plan was to use one middle-ware server for every balanced mirror, not every tor instance.

A bad decision lead to an IP leak of the middle-ware server.

Versus was down for about 14 hours to move our complete infrastructure to other hosts. From every DC, not just the compromised one.

Some additional errors then popped up because bitcoin-core, monero-core and our database got updated to the most recent version.

How we fucked up.

A trace route showed a perfect loop-back on the middle-ware server IP so the request was routed there via IP as nothing would go beyond the first router.

THIS WAS A BAD DECISION MADE AFTER NEARLY 3 DAYS WITH NEXT TO NO SLEEP.

The middle-ware gets the onion and adds it to a custom header so that we see from which mirror the request is coming from, but instead of extracting the onion, it extracted the IP from the request (it’s own) and added it to the header.

The main server used this header to generate the 2FA message where the IP was shown.

What we did to avoid such stupid mistakes in the future.

The middle-ware was fixed to extract to correct onion under any circumstances.

The display of the hostname gets validated via regex.

There is no more routing via IP. The middle-ware will run on every single tor instance. The routing to the back-end server is also over Tor.

Additionally we checked the source to look for any other instance where the hostname/custom header might show up. Nothing was found.

This multiple layers of security will make sure, that even a bad decisions under sleep deprivation will not lead to such an event again.