The Tor BSD Diversity Project (TDP)

BlogFrequently Asked QuestionsResourcesGitHub RepositoriesContactTDP Hidden Service

The TDP Projects:
Tor Browser for OpenBSDBSD Tor Relay GuidesBSD Firm RelaysPorting Targets for PETsQuick-and-Dirty Static Reports

A Blog, or a Central Location for Announces and Notes

March 2017

Amsterdam in March

February 2017

Running OpenBSD -current for Tor Relays?

December 2016

Welcome Aboard, Vinicius

November 2016

In The Tree

October 2016

More Bananassignify(1) SanityTest Tor Browser 6.0.5 for OpenBSDReplying to Tor Blog Comments

September 2016

Where Things Stand

August 2016

The Buildbot Needs BSD Relays

June 2016

So Much Quiet Progress

March 2016

TB 5.5 on the Current SnapshotsPorting PETs UpdatesMarch 10 OpenBSD SnapshotsTDP at BSDCan 2016TB 5.5 and i386/amd64 Snapshots

February 2016

TB 5.5 and More New SnapshotsTB 5.5 and New SnapshotsOnto the Next PhaseTor Browser ReleasesTor Browser 5.5 Ports Tagged

January 2016

Still Plugging AwayTor-in-a-Box?

December 2015

Notes From The FrontAnnouncing Porting PETsThinking About 2016

November 2015

PETs Porting TargetsThe Case of the Brazil RelaysTB 5.0.3 Packages Updated, AgainComing Soon: Quick-and-Dirty ReportsThoughts on Reproducible BuildsIt’s Up to You

October 2015

Updated Tor Browser PackagesThe BSD Relay GuidesOur First BellsBeyond OS DiversityTor Browser version 5.0.3 for OpenBSD

From the Attic

20170317

March in Amsterdam by gman999

For a variety of reasons, only one of us will be attending the upcoming Tor Summit in Amsterdam. If you are attending, please make sure you speak to Vinicius.

Huge thanks to torservers.net for the last minute support for him. Enormously appreciated.

20170227

Running OpenBSD -current for Tor Relays? by gman999

The question of which branch or flavor of OpenBSD to use for a Tor relay is a frequent point of mention.

OpenBSD maintains three flavors:

The logical notion is that -stable or even -release should be the Tor relay platorm choice. It does seem to be the most common recommendation.

Yet there are a number of things to consider, and we tend to favor -current as the best option in most use-cases for any OpenBSD box.

First, what is -current in OpenBSD is not some wildly unusable system. -current is the platform for OpenBSD development, in that it is the flavor on which OpenBSD developers actually work. Not a few production servers run on -current, and most significant problems are quickly resolved.

The other issue to consider is that OpenBSD’s ports development takes place on -current. Therefore the most current OpenBSD ports are found in -current. net/tor is at version 0.2.9.9 with a single revision, while -stable is still at Tor 0.2.7.6 with three revisions.

And no, for the inquisitive, OpenBSD does not support alpha or beta software in its ports tree, which excludes the Tor development branch.

Updating -stable isn’t difficult, which ever updating routes chosen. But following -current with the regular snapshots is equally simple. This guide from Peter Hansteen is dated, but gives the gist of the procedure.

Just because -current can update as frequently as a few times a day at times, doesn’t mean the Tor relay operator has to update the system each time. If one can keep a -current relay updated weekly, it should be fine.

The one other thing to note is that physical or serial console (or similar) access is also necessary for updating -current, as one has to boot off the bsd.rd kernel.

20161217

Welcome Aboard, Vinicius by gman999

Two of us launched TDP in March 2015. The accomplishments are substantial, and the impact is significant. TDP did more than raise the PETs flag in BSD land, it sparked a number of related projects and efforts.

One important player in the broad effort has been Vincius Zavam, a young BSD hacker in Brazil. Introduced to PETs and Tor in particular as he was an organizer of BSDCon Brasil in 2015, Vincius wasted no time engaging on multiple levels.

He not only started running a number of Tor relays on several BSDs, he also assisted others in Brazil. Without any external assistance, he began porting Tor Browser to FreeBSD, an enormously important endeavor. And most recently, he did a presentation on Tor with a focus on the BSDs.

He is not just full of energy and ingenuity, he is also a pleasure to work with. His locale of Brazil is another advantage which he has utilized to the fullest. He has provided patches to the FreeBSD security/tor port several times and is active on the Tor-BSD mailing list.

Like the other TDP members, he might favor one BSD over another for different tasks, but “BSD agnosticism” is his ideology. Although this tweet might belay his deepest allegiance.

Welcome aboard, Vinicius. Our efforts are ongoing without monetary compensation. There is no pot of gold awaiting us, but if we can continue to enhance Tor and the PETs scene in general, the satisfaction is priceless.

20161113

In The Tree by attila

It was a long haul but the Tor Browser ports were finally accepted into the OpenBSD ports tree today.

A huge thanks to landry@, semarie@, danj@, mmcc@ and all the other ports@ people who made this possible with their critiques, observations, pointers, suggestions and just plain work.

Now we’ve got to get cracking on Pluggable Transports. We won’t stop not stopping.

20161025

More Bananas by gman999

The easiest way to convey the problems with monocultures in technology is to point to a stark monoculture common to many: bananas.

The Cavendish banana is a non-reproducing variant that dominates global production, and is most likely the one encountered. The Guardian published an article entitled “The banana as we know it is in imminet danger” regarding efforts to diversify banana types.

20161017

signify(1) Sanity by gman999

signify(1) was released in 2014 as a simple means of signing and verifying files on OpenBSD.

What does signify have to do with TDP?

The current and standard practice for verifying the integrity of digial signatures on software is to use GnuPG. The procedure is tedious, and likely ignored by most TB users.

signify is small and sticks to the Unix approach of doing one function.

20161005

Test Tor Browser 6.0.5 for OpenBSD by gman999

The release of TB 6.0.5 has been worked and reworked, and submitted to the OpenBSD ports@ list.

If you’re an OpenBSD user running amd64 snapshots, and you want more operating system on the Tor network, you should be testing it. As always, consider this release without warranties and a serious infraction to your online anonymity!

The amd64 packages are on the NYC*BUG mirror. The i386 packages should be up in the next day or so.

Feedback, comments, gripes not just welcomed, but demanded.

20161004

Replying to Tor Blog Comments by gman999 and attila

The last Tor blog post on Tor 0.2.8.8 is released, with important fixes prompted a flurry of comments regarding the BSDs and the Tor Project.

An important part of the Tor 0.2.8.8 release was a fix for bug #20103 discovered by TDP. Tor on OpenBSD was crashing with OpenBSD relays running 0.2.8.7 as the first hop. Gman999 first encountered the crashes while testing new TBB packages. Attila did the heavy lifting and reported it to the TP’s Trac.

OpenBSD is an ideal bug-finding platform as it follows the classic Unix approach in which a daemon dies loudly rather than quietly hiding its behavior. The bug likely affects other operating systems, so another +1 for operating system diversity.

The comments section opened a noisy series of posts about Tor and the BSDs, some of which we believe are inaccurate and demand responses. Snips from the posted comments are below, with our replies:

Since nickm mentioned OpenBSD users have been more seriously affected, we’d like to take this opportunity to ask why The Tor Project has no plans at all to release Tor Browser Bundle for *BSD operating systems, OpenBSD in particular.

There are lots of things the TP should be planning, and as non-Tor Project developers, we jumped on the opportunity to port Tor Browser to OpenBSD back in March 2015. We are in regular contact with the TP, and have been encouraged and assisted by a number of TP core people, including Moritz and Roger. Gman999’s recent attendance at the Seattle Tor Summit illustrated the great attention TDP is getting; he was personally flattered by the recognition.

The question for the anonymous poster is “what are you doing?” TP is an open source project. In BSD Land no one listens to gripes about software if the complainer doesn’t at least begin resolving the issue, like submitting debugging information, providing a patch, etc. It’s a principle that all open source projects should adopt. The poster in question should at least be testing our TB releases on OpenBSD. Fork the code, submit a useful issue, and so on, but complaining anonymously on a blog about what others should do is pointless at best.

It appears then that The Tor Project is not keen at all to support users of *BSD operating systems. Therein lies the danger. Again the following is a quote from The Tor BSD Diversity Project…

This comment follows up from noting the lack of source tarballs from the TP, which is the preference for porting Tor Browser and other software. Oddly, the poster quotes the TDP www site, yet in the previous comments says the TP “has no plans at all to release Tor Browser Bundle for *BSD operating systems.”

Nick M, Roger D, and others actively use the BSD Buildbot initiated by Christian S., and we have corresponded about it. Roger made multiple references to TDP in his postings on various Tor mailing lists. We hardly feel there is some conspiracy against the BSDs from Tor core developers. Rather, there is a genuine recognition of the TDP endeavor.

More generally: yes, the recent vulnerability from RFC5961 TCP implementations on Linux makes yet another strong case for operating system diversity, and it is neither the beginning nor the end of screams for diversity.

And it’s not just operating system diversity. More relays need to be running on architectures other than x86. PPC and ARMv7 are certainly worthwhile platforms, and the near-future should see production-quality support of 64-bit ARM hardware (aarch64) on FreeBSD.

Moroever OpenBSD users prefer to download Tor Browser Bundle directly from The Tor Project as the latter is the official software publisher of Tor.

We doubt too many open source operating system users would prefer to directly download from any third party for their packages as opposed to using software packaged specifically built for their operating system in their respective ports or package system.

In the case of OpenBSD, this is doubly true. If software gets into OpenBSD’s ports tree, a small but real amount of review is conducted at minimum. Third-party ports in the tree aren’t fully audited, but as one can see from the ports@ mailing list, ugly unreviewed ports don’t easily enter. We have been developing and tweaking TB on OpenBSD for a long time. Maybe if we could attend to the porting effort with more time and resources, TB would already be in the OpenBSD ports tree, but regardless, look through the comprehensive attention TB has received. Moreover, development on OpenBSD only happens on the -current branch, which changes rapidly and frequently.

There is another issue that gets glossed over when people propose that the Tor Project should distribute OpenBSD TBB packages: who signs them? It is not that common in the OpenBSD community for users to have umpteen keys from umpteen software repositories installed in /etc/signify and to mix packages installed from various sources. Users generally install packages built from ports by the OpenBSD team, signed with the keys distributed with the operating system (one notable exception: M:Tier, which is run by OpenBSD developers). We are therefore hesitant to suggest that the Tor Project start distributing packages, since they would then have to sign them and instruct their users to add the appropriate keys to their system. The better way, in our opinion, is for the port to be accepted into the official ports tree and for binary packages to be made available in the usual way.

If I had seen all this sooner, I wonder if it would have been worthwhile to have suggested tackling *BSD Tor packages be a hack topic at the recent Tor Project Hack Day in Seattle this weekend? Perhaps there’ll be a future chance?

It wasn’t an explcit topic in Seattle, but there was a fruitful discussion about diversity, and not just operating system diversity. There were also a number of informal discussions on the topic, but the ball is really in TDP’s court right now. TDP will need more TP involvement in the near future.

Regarding a version of TAILS on a BSD: It will never happen as *BSD kernels are notorious for being behind in their support for the latest Intel CPUs.

This is really just FUD: the vast majority of hardware is well-supported by the BSDs. OpenBSD is refreshingly intransigent about signing non-disclosure agreements, which can mean lack of support for some hardware, but they do distribute firmware for some wireless cards and other devices. Without that stance, a lot of open source development would never occur. “Open source operating system” would like mean a collection of mysterious bloated binaries with a “Certified Open Source” sticker slapped on the product.

Producing a TAILS-like alternative based on OpenBSD has been a goal of TDP since its inception. We’re still on the first step: porting TBB to OpenBSD.

And again regarding the RFC5961 issue in Linux: the argument should just be about having diverse kernels, because saying a known bug “proves” one of them to be inherently inferior is actually a temporary fact. In using a diversity argument and avoiding a comparative argument, I’d expect more support for *BSDs will be attracted from thinking people.

We aren’t focused on ‘superiority’ although most people who choose an operating system logically consider it superior on some level or another.

It is generally true that each of the BSDs have different roadmaps for implementing standards than Linux. In many cases, the BSDs differ among themselves, despite a shared origin and lots of overlap since.

I don’t understand why you brought Apple into the discussion.

and,

I think the OP refers to the public secret that Apple pilfered one of the *BSDs and adapted it into iOS, because the generous nature of the BSD licence allows it, hence why “BSD things are more better for closed source” as the OP puts it. It’s well known that iOS is a Unix clone, at least.

In a sense the BSD license throws up a white flag in the face of corporate usage of code. It’s a pointless battle, and their side always has more resources and lawyers. The point of the BSD license is to protect the developer, and to let code flow around like it should without restrictions; the simplicity of BSD-derived licenses is impossible to deny. We view licenses like the GPL (esp. post-v2) as generators of billable hours for corporate lawyers, something we’d care to avoid.

Apple does use BSD code, as do many other firms. “Pilfered” hardly describes the relationship. And yes, that lack of restrictiveness does mean BSD code is used but not loudly. Having a license that developers don’t need a lawyer to read offsets the loss, and BSD-licensed code’s influence is deep.

That’s it for now.

20160906

Where Things Stand by gman999

A brief list of where things stand with TDP:

That’s the status right now. Updates should be happening in the near future.

20160818

The Buildbot Needs BSD Relays by gman999

One of the small yet important projects spawning from this Tor-BSD meme is Christian S.’s BSD Buildbot. Essentially it’s a tool for development builds of Tor for the Tor Project, with volunteers enlisting their BSD relays.

Recently its relevance was reinforced due to some libevent issues with OpenBSD and Tor. The OpenBSD base includes libevent, and libevent2 is a dependency port (LIB_DEPENDS) for the Tor port. Tor Project Trac tickets include 19902 and 19904.

To enlist a BSD relay in the buildbot:

$ buildslave create-slave slave buildbot.pixelminers.net:9989 <buildername> <password>

where is something you choose, and is your key for the particular slave.

$ buildslave start slave/

For OpenBSD buildbot relays, /etc/profile needs to list the installed versions of autoconf and automake, as per this email to the Tor-BSD mailing list.

export AUTOCONF_VERSION="2.69"
export AUTOMAKE_VERSION="1.14"

Assuming everything is configured correctly, the buildbot slave should appear on https://buildbot.pixelminers.net/buildslaves, and the log in slave/twistd.log should provide results.

Also: yes, we think the terminology of buildbot “slave” and “master” are inappropriate, and we only use them since they are the actually commands and lingo. The terms are not even descriptively useful to someone new to the concept of a continuous integration system.

20160629

So Much Quiet Progress by gman999

This blog remained silent over the past several months despite a flurry of very significant TDP activities.

The accomplishments are not reducible to a single blog entry, but here are summaries of the more interesting:

We would have replied to the RFP for EuroBSDCon 2016 in Belgrade, Serbia but the TDP schedule doesn’t permit.

20160331

TB 5.5 on the Current Snapshots by gman999

As of last week’s OpenBSD’s i386 and amd64 snapshots, TB 5.5 is no longer working.

We are looking to start building the Tor Project’s most recent TB soon. Spending time on TB 5.5 is fruitless when 5.5.4 is the current TP release.

The OpenBSD project just announced the release 5.9 a month early, which I personally don’t remember ever happening. The project usually follows a strict six-month release cycle. We are going to focus on getting TB into the next stable release of OpenBSD, which would be 6.0, planned for November 1. Of course we hope to have TB in the snapshots ports way before that date.

Meanwhile, the Quick-and-Dirty Static Reports are still updated regularly, albeit manually still.

Additionally, a lot of time has been recently committed to Porting Targets for PETs. It’s a tough battle. You spend time getting the basic aspects of the Makefile operational, you figure the peculiarities of how a port is compiled, the array of licenses, and so on, but then you realized a host of unported Python libraries build or run dependencies. Jump out of vi(1) and into the rabbit hole.

A final note on porting PETs-related software, mostly directed at developers. Write your software to be portable, please. Creating a Python module port or package may be a simplistic example of portability, but a negative example is doing builds specific by each OS and Linux distribution. Don’t give me setup_debian.py, or a setup file that relies on a handful of operating system choices. Give me an install script that can recognize the global variables and avoid hard-coded paths, that doesn’t need one shell or another. What does bash provide that the install script requires? I mean, really? For the vast majority, a 1995 Bourne shell would be more than sufficient.

If you only want, say, Debian users, for your PETs application, you are definitely not looking at the basic diversity arguments so apparent to most people. You are also cutting off more potential downstream developers that could be making your life easier. Treated nicely, downstream developers can make you look significantly smarter than you might be. There are arguments whether the “many eyeballs” make open source software more secure when most people don’t read code, but there’s no question that more downstream developers hacking on your code really does.

20160320

Porting PETs Updates by gman999

Porting Targets for PETs is meant as a hit list of privacy enhanncing related ports that may or may not be in the main BSD port/package systems. As we make clear on the page, some should be considered for entry into BSD ports systems, while others may not be good candidates for a variety of reasons.

As we (manually) update the project page, a few points come to our attention:

Don’t hesitate to ping us if you’re interested in addressing anyone of the above ports. We can direct you to the appropriate mailing list or developers, or dump any Makefiles we may have already created.

At this point, TDP isn’t focusing on porting these applications, but unofficially we have begun to toy with some of them.

20160310

STATUS: TB 5.5 and i386/amd64 Snapshots by gman999

Tor Browser 5.5 is still working with the newest OpenBSD snapshots, which is #1638 for i386 and #1918 for amd64.

20160303

STATUS: TB 5.5 and i386/amd64 Snapshots by gman999

TB 5.5 is working fine with the newest OpenBSD snapshots:

20160301

TDP at BSDCan 2016 by gman999

Our presentation entited “Beyond Monocultures” was accepted for BSDCan 2016 on June 10–11 in Ottawa, Canada.

We feel very fortunate, since there were a lot of submissions for what is the premier BSD event in the western hemisphere. BSDCan has grown substantially since 2004, with hundreds of attendees participating.

Last year we conducted a birds-of-a-feather session which attracted around 50 people.

This year, we should hopefully have some good news to publicize about TDP. In the meantime, the actual presentation should start coming together in the next month or so. In all honesty, it’s hard to determine what the focus of the presentation should be. We only have a general idea of where we’ll be by mid-June.

At this point, the introductory part should start with “Why Tor Sucks” plagiarizing Henning Brauer’s OpenBSD Sucks concept. This is a backhand approach to arguing why Tor matters. The reality is that a lot of people in the BSD community continue to see Tor as ineffective or insecure, so the case for Tor being full of problems yet the very best thing available needs to be made.

It’s not that BSD people overwhelmingly aren’t concerned with security and privacy. At the bar last at BSDCan last year, surrounded by some of the best known BSD veterans from the 1970’s, I listened how they dealt with surveillance. One file system hacker with decades of experience mentioned how he always changed his MAC address when using public wireless.

But there’s also a sense that Tor is too easily broken, such as with timing attacks from a global passive adversary. Or that if enough Tor relays are run by an adversary to anonymity, the network is useless. All are valid points and certainly need further attention. The heavy brains at the bar that night could infuse some assistance.

20160223

TB 5.5 and More New Snapshots by gman999

Yesterday the OpenBSD snapshots updated to #1881 for amd64 and #1609 for i386. TB is working fine on both.

Meanwhile, the TB packages were updated. First we implemented a meta TB package, as per landry@’s comments. Note that the start-tor-browser script was also deprecated recently. All the sloppy setup gook was replaced by neater Firefox hacks.

Installing TB from the mirror is simple:

$ doas env PKG_PATH=http://mirrors.nycbug.org/pub/snapshots/packages/amd64/ pkg_add tbb

For i386 installs, replace /amd64/ with /i386/.

Be aware that that we are still tinkering with TB 5.5 which has some significant vulnerabilities that could disclose a user’s identity. TB 5.5.2 is in the pipeline.

A question for TB testers out there: does a Tor Browser icon appear on the desktop after TB installs?

According to the general standards on window manager desktops, it should as the installer places /usr/local/share/applications/tor-browser.desktop into ~/.local/share/applications. However, on XFCE it doesn’t appear, and the file needs to be placed in ~/Desktop. How about KDE and GNOME users out there? Let us know.

We imagine that most users following -current are probably running cwm.

20160221

TB 5.5 and New Snapshots by gman999

Just a short note: Tor Browser for OpenBSD 5.5 is still working with the most recent OpenBSD snapshots (#1880 on amd64 and #1608 on i386).

We always use the most recent snapshots on our boxes, and usually update the TB packages when TB needs updating due to relevant snapshot changes. Since our build process significantly simplified with TB 5.5, primarily due to landry@’s input, updates to both amd64 and i386 builds became relatively painless.

And it should only get better in the near-future releases.

20160216

Onto the Next Phase by gman999

The progress we’ve made over the past five days was exhausting, yet exciting.

Some very significant steps were made with Tor Browser. A revised 5.5 release does not contain the start-tor-browser script any longer; all the necessary setup steps are now done with Javascript, including the profile setup. The Firefox add-ons are now dumped into the profile as files, such as https-everywhere@eff.org.xpi, instead of being extracted into directories. Additionally, we are now building an i386 version of TB.

We are aware that TB 5.5.2 was released by the Tor Project this past Friday, a mere hour after we announced our 5.5 release. TB 5.5.2 includes some important security changes coming from the Mozilla upstream, although much can be mitigated by moving the security slider to high. And TB 5.5.1 was also released before our TB 5.5, but the changes were significant. Nevertheless, the last five days of constant hacking and testing on TB makes future releases less painful and more smooth.

Finally, for those doing TB testing, we have a brief guide to what to test in a Testing Tor Browser piece. The past week has already changed some of the steps, and we look to expand and formalize this document so it becomes a useful tool.

It’s almost a year since our first commits to GitHub, and it’s been a long and sometimes painful learning experience, but we think we’re now in a great spot.

20160205

Tor Browser Ports for 5.5 Tagged by attila

I just merged 5.5 onto master and tagged it. This release was much easier after the work on 5.0.6, which has us using mozilla.port.mk instead of a bunch of cut-and-paste adapted from same. This makes things a lot easier moving forward. So far 5.5 on amd64 is looking good.

Tor Browser Releases by gman999

It’s been a while since our last Tor Browser releases, but it’s not because we haven’t been busy. Smaller projects like Quick and Dirty Statistics and Porting PETs have continued to progress, and other stuff that attila can elaborate on in a future blog post.

While worked dragged on with the 5.0.6 Tor Browser release, we managed to not only finish that release, but also finish up the 5.5 release. That puts us parallel with the current Tor Project release.

We are now archiving previous versions of TB as tgz file in an an archive directory. Version 5.0.6, which barely saw the light of day, is there.

We are excited by the releases, and look forward to feedback from the testers out there.

One quick note about Tor Browser 5.5 on OpenBSD 5.8 stable. We have repeated ad nauseum, but it’s worth reiterating again: OpenBSD development happens on -current, a.k.a., snapshots, which ultimately turn into the next stable release every six months.

Developing Tor Browser for OpenBSD stable means dealing with multiple levels of differences between stable and current, from the package versions to libraries in the base operating system. In *BSD land, userland applications and the base OS are meant to play nice together by default.

In the case of trying to run TB 5.5 on OpenBSD 5.8 stable, there are two package incongruities. First, 5.8-stable’s Tor version is 0.2.6.10p1, while the current version is 0.2.7.6. Second, the nspr stable version is 4.10.8, while the current version is 4.11. Both current versions of the packages rely on operating system changes not present in stable. There is a ports freeze approaching for the May 1 OpenBSD 5.9 stable release, and we may use that as an opportunity to produce a stable version of the Tor Browser. Stay tuned.

20160118

Tor-in-a-Box by gman999

If someone reasonably technical bumps into Tor for the first time, eight or nine seconds later, they arrive at the concept of some type of Tor device that automagically routes all local network traffic through the Tor network.

Great idea. The fact that so many imagine such a concept certainly means something.

All too often, unfortunately, the implementation is wrong. Dead wrong.

In the early 1990’s, a desktop’s traffic to the public internet was simple. There might be some HTTP from a web browser, a dash of UDP for DNS lookups and maybe some POPing to a remote email server. All was relatively quiet.

Over the past two decades, the wall between the internet and the desktops evaporated. Why is Windows 10 a free upgrade? Likely because a “free” operating system is well-compensated by full control of the desktop environment. And that means a tcpdump(8) from 1994 bears no resemblance to the ugly spew of 2015.

After those initial eight or nine seconds, going back to the basics of design should be the next reaction. Stop trying to make tools that do everything half-way. Too often an all-in-one device that tries to solve multiple problems displays contradictions between those functions. Thus, the core Unix principle that is considered dated by many, yet justifies itself with each new “wonder box” incarnation: one tool for one function.

That is not an argument against innovation, products or progress on any level. The point is the moment an attempt is made to cast a wide net into a complex sea, the net is shredded. The net’s target is not a school of sardines, but sardines, sharks, with rusted earth-moving equipment, yellow school buses and maybe a piece of space hardware or two.

More on this theme later, but the resilience of the classic Unix themes rises from the grave daily in an age of all-in-one products or services that tries to do everything, but succeeds at doing some things poorly and others dead-wrong.

20160102

Still Plugging Away by gman999

We are still moving along.

Attila began working on the next Tor Browser release yesterday.

Dirty Statistics are still being updated and tweaked. More reports are in the pipeline.

Some inferences from the Dirty Statistics reports:

Remarkably, Tor Browser 5.03 is still functional on OpenBSD/amd64 with the #1783 snapshot from December 27th. Snapshots frequently take hard twists and turns, as is to be expected with the development branch of any operating system, so this is something of a surprise. The early releases of TDP’s 5.03 faced some hiccups with various changes, but we are trouble-free since.

One thing to note is that the number of public *BSD Tor relays, not including BitRig, remains consistently above 5% of total relays. While we can’t necessarily attribute to TDP, we like to think the noise we make helped a little bit.

Stay tuned. We are still very active, even when we are publicly quiet.

20151216

Notes From The Front by attila

First: hats off to gman999 for his incessant efforts in getting the content of this site in better shape. I especially applaud this low-tech/no-tech blog layout in MultiMarkdown.

I have been noticeably lacking, but not totally idle. I’ve had to take some paying work, which has slowed me down on open source, but my path forward is fairly clear. My main task is to rework the makefiles (mainly the stuff in Makefile.inc) that comprise the OpenBSD ports for TBB so that they dovetail with and use as much as possible of the Mozillan infrastructure already in the OpenBSD ports tree, much of it due to landry@, who has already helped me a couple of times. I should’ve done this from the beginning but my head wasn’t really on straight when I first started this. I’ve been reticent about touching anything that I didn’t write, choosing instead to adapt what others have done to get something working. Although this was perhaps effective in the short term if we want this in the tree it has to be consonant with it… in short: if you’re serious about contributing to OpenBSD then pick up a shovel and start digging, but politely. I’m sure I can do that so I just have to get to it.

I hope to have a first cut at a rework of the ports, still based on 5.0.3, sometime next week… I don’t really celebrate any holidays so I’m hoping to get a lot done while the rest of the world sleeps it off. Once the makefiles are closer to right I’ll work on an update to 5.0.5 (or whatever is current on the 5.0.x branch). I’m afraid I might miss the next ports lock window because I’ve taken too long, but oh, well… que sera sera.

Announcing Porting PETs by gman999

One of the small projects we have spent some time on recently is Porting PETs. This is an attempt to list the various privacy-enhancing applications and their statuses in the BSD ports.

Most of these ports arose as non-commercial, open source reactions to mainstream applications and services. Some are ported to one BSD or another, others are not.

The list is not exhaustive, but it was certainly exhaustive to create. Updates will happen manually, so diffs are appreciated.

Porting third-party applications is a frequent gateway for BSD users to become developers, this list will be circulated in the relevant BSD channels.

20151202

Thinking About 2016 by gman999

The BSDCan 2016 call for papers was issued yesterday, and a TDP-related submission was made. BSDCan is likely the largest BSD gathering globally, and an excellent opportunity to speak to *BSD developers and users.

EuroBSDCon 2016 is tentatively slated for September 2016 in Belgrade, Serbia. It is another significant BSD event, attracting users and developers from Europe and beyond. At a glance, there are only two Tor relays in Serbia, and both are Linux. Beyond Serbia, there are few Tor relays in the Balkan states overall, making EuroBSDCon 2016 a great opportunity to extend not just BSD Tor relays, but any Tor relays.

No dates have been set for AsiaBSDCon, but it’s usually in March. Japan is well-wired with inexpensive residential broadband, yet there are only around 50–60 relays in the country. Considering it’s a BSD-heavy nation, it’s shocking that there are only a handful of *BSD relays. Yet another green field of opportunity.

Stay tuned. Whether we can speak at any of these events will also depend on financial support for TDP.

20151119

PETs Porting Targets by gman999

After the June 2013 Snowden disclosures, a rush towards developing applications to counter mainstream, closed-source services commenced. Many focuse on Debian Linux as a development platform, but aim at more widely used Windows, OSX, iOS and Android user-base. Beyond client applications, there are also network-based servers and services seeking to provide privacy and anonymity.

The term “PETs” refers to privacy-enhancing technologies, and in this case, we use it as a catch-all for these server and client solutions.

Some of these projects have been ported to one or more BSD. Others have not. On that note, we began a list of applications and their status as BSD ports in the main BSD operating systems. We encourage feedback on this list, and also investigations into porting these applications. Some of well-worth reviewing and considering; others have ceased development or are broken beyond resurrection. Others just need some reworking towards sanity, as one will notice that ubiquitous build dependency bash.

It’s a call for engagement to the *BSD community. Bring your sane, portable development approaches, your intransigent working and reworking of Makefiles, your austere mentality. This is an opportunity to improve applications whose user base might be someone whose life depends on it.

20151112

The Case of Brazil Relays by gman999

Just a short note about the Brazil relays implemented after BSDCon Brasil. Before the early October conference, it seems there was only one public Tor relay. After the event, there are up to five or six relays maintained by two separate individuals. It’s understood that some bridges also joined the Tor network also, but we don’t have any direct evidence.

It’s not an enormous leap in relay numbers for Brazil, but adding a handful of relays to a country that only had around 20 is significant. More importantly, it seems that the new BSD relays contribute a decent amount of bandwidth.

However, there is a discrepancy between the “Tor Status” data provided by https://torstatus.blutmagie.de/ and https://torstatus.rueckgr.at. Part of this might be answered by the fact that both are statically updating at different intervals. Olaf Selke of BlutMagie.de, also notes that the “observed bandwidth” numbers are diffferent since only BlutMagie.de shows the average bandwidth based on the “extra-info” descriptor while other sites, including Rueckgr.at, display the peak bandwidth.

However, there still seems to be discrepancies beyond the “observed bandwidth” field, which we will look at further in the future. For instance, a bunch of AWS Tor relays appeared on BlutMagie.de the other day and remained for a few days, but never showed up on Rueckgr.at. And more oddly, all the relays quickly remained highest bandwidth providers for a few days, in contrast to the normal trajectory of a relay.

For now, either site is useful for giving a broad picture of the public Tor network. It is true that the Brazil relays look at a lot more significant in terms of observed bandwidth at Rueckgr.at.

20151111

TB 5.0.3 Packages Updated, Again by gman999

The Tor Browser 5.0.3 packages were updated again, due to the need for icu4c version 56.1. Both devel/nspr and textproc/icu4c are updated in the OpenBSD ports tree, and the TB packages have been rebuilt for them. Be sure to make sure packages or ports are updated before installing. If the host is updated, there’s no need to use our devel/nspr or textproc/icu4c packages.

This is all a true story of the constant attention necessary to develop sanely on any operating system: develop on the most current version, and look forward to an automated build process for it, but ultimately a stable version is a beautiful thing.

20151108

Coming Soon: Quick-and-Dirty Reports by gman999

This week we’ll post “Quick-and-Dirty Reports,” providing diversity-related snapshots of the public Tor network. Currently, the five reports are generated manually from the ruckgr.at Tor Status CSV files, but they are being migrated to SQLite in the future.

A quick sample of the current reports were posted in an earlier blog entry.

The only comparable service we know of is the Tor Metrics site which has the additional function of providing historical data. Our goal is considerably more mundane, yet also functional.

Nothing ground-breaking or revolutionary about the reports, but we do hope others find them useful, and maybe event extend their use.

20151104

Thoughts on Reproducible Builds by gman999

Just a quick link to a pleasantly polemical post from September 19th by OpenBSD’s tedu@ entitled reproducible builds are a waste of time. There’s a follow-up postscript at the end of the post, reacting to a lobste.rs thread.

20151103

It’s Up to You by gman999

Since we launched TDP, two of us spend a lot of time, energy and resources getting the various projects designed and implemented.

But there’s is always room for one, two, three more.

It’s a perfect opportunity to start testing Tor Browser 5.0.3. Fork the repository, submit issues about the software.

There’s more to do beyond TB though. If you are around a technical user group, get a discussion going about Tor. Have a how-to meeting on running a public relay, especially for those who have access to decent infrastructure and bandwidth, like those at universities or internet-facing firms.

Setting up a high-bandwidth Tor bridge is painless and it will just be a safe gateway for Tor users.

There is no excuse why every BSD user group or conference shouldn’t have a discussion or session focused on “recruiting” BSD people to run Tor relays. Many people in the US, Europe and eastern Asia, in particular have excess bandwidth at home. Work at a firm that uses the BSDs in production? Get them to run a relay or two.

For those who dwell in BSD land, join the Tor-BSD mailing list. Running a BSD Tor relay? Join the unofficial BSD Buildbot for Tor.

There’s a lot we’d like to accomplish with TDP, and we don’t claim a monopoly on much of anything. We do encourage you to take some initiative and move things forward.

20151031

Updated Tor Browser Packages by gman999

The upstream code from the Tor Project and above them Mozilla is a moving target we contend with each release. Then there is the ultimate moving target: the incessant war between surveillance and anonymity, censorship and circumvention. Finally there is the operating system as a moving target all Tor Browser porters face.

Developing ports on OpenBSD means building on snapshots, a.k.a., -current. OpenBSD snapshots are often released several times a week, and as with any other development operating system branch, those changes are sometimes significant. What might work today may be broken tomorrow. It was no surprise that our TB 5.0.3 release was broken on the OpenBSD snapshot released just a few days later.

Daniel Jakots noticed this as we did, and we updated the Tor packages accordingly. We made an added change by removing the tbb meta-package, simplifying the 5.0.3 release a bit more.

20151030

The BSD Relay Guides by gman999

In their current forms, the BSD relay guides are unclear and sloppy, and possibly inaccurate in some places. We are putting some work into the FreeBSD Guide for Configuring Relays, and will probably divide it into two parts: the short version for the impatient, with other topics being migrated to another page entiled “Advanced.”

There is a lot of related topics to cover: to ZFS or not, slimming down a FreeBSD build, etc.

Input is welcomed. Stay tuned.

20151029

Our First Bells by gman999

Over six months ago we launched TDP in its current form. In March, the GitHub repository was initialized and we put some meat on the skeleton we had been toying around with.

We count a lot of accomplishments since launching, but should be honest about TDP’s weakest point: marketing and publicity. Of course, it’s something we’re proud of to an extent. Too many open source projects focus almost solely on publicity, and don’t actually accomplish much else. Nevertheless, we will try to start providing a clearer picture of our progress and notes here.

Quite frankly in BSD land, all fluff and no product doesn’t get you much credibility. Talk is considered cheap, while good code and real contributions are priceless.

We announced the sixth release of Tor Browser two days ago, version 5.0.3, which was a major milestone for us. We were excited by some of the feedback on the Tor-talk.

The presentation at BSDCon Brasil was a success, with the first BSD relays launched in that country, and which now account for up to a third of observed Tor bandwidth there. More relays should be coming online in Brazil soon, and we know of a few BSD bridges that came online.

Interestingly, one of the relays running on a residential connection, had to be migrated to a bridge. Apparently, one of the relay admin’s online services blocked the relay’s IP. The IP wasn’t an exit relay, but it was a public Tor IP. Let’s hear it for collective punishment.

Finally, we will be catching up with the publicity we should have previously done in the near future. Bear with this blog’s format, we are not really the WordPress types, if that wasn’t already apparent.

20151028

Beyond OS Diversity by gman999

TDP focuses on operating system diversification for Tor with BSD Unix. But the need for diversity is more than just operating systems. A quick browse at one of the Tor Status sites, or more specifically the Network Statistics Graphs, the lack of geographical diversity is disturbing.

Parsing the list of Tor relays, there are a number of ISO 8166–1 two-digit country codes that have no relays. Spreading the Tor network out to those countries should be a primary concern, regardless of operating system.

We don’t know the particulars of infrastructure, connectivity costs, etc., in a lot of those countries, but the underrepresented regions need a dedicated focus.

Also disturbing is the concentration of public relays by country code. Germany and the US contain more than a thousand relays each, accounting for more than a third of the total number of Tor relays globally.

Of course, using Tor in some of those places may be dangerous or cost prohibitive.

We will continue to tinker with the data about geographical diversity in the future, but in the meantime, if you have contacts, friends or families in the underrepresented country codes, now is the time to explore the possibility of getting Tor relays into the Tor-less country codes.

20151026

Tor Browser version 5.0.3 for OpenBSD by gman999

The Tor BSD Diversity Project (TDP) is proud to announce the release of Tor Browser (TB) version 5.0.3 for OpenBSD.

TDP is an effort to extend the use of the BSD Unixes into the Tor ecosystem, from the desktop to the network.

The 5.0.3 release is the sixth release of the Tor Browser from TDP.

To install TB for OpenBSD, please see http://mirrors.nycbug.org/pub/snapshots/packages/amd64/README.

TDP is focused on diversifying the Tor network, with TB being the flagship project. Additional efforts are made to increase the number of *BSD relays on the Tor network.

Since its launch in March 2015, TDP contributed significantly. In addition to the TB releases, both BSDCan and BSDCon Brasil featured TDP-focused meetings.

In early October, a TDP presentation prompted a significant increase in Brazilian Tor relays. Before the presentation, there were around 22 Tor relays, all of which were Linux in addition to two Windows nodes.

In the weeks after the presentation, several new *BSD Tor relays appeared, accounting for up to 35% of observed Tor bandwidth in Brazil.

Finally, TDP is working to convince BSD-using businesses to follow Mozilla’s December 2014 example to run Tor relays. At this point, New York Internet has committed to running two high-bandwidth relays, one FreeBSD and the other OpenBSD, at its facility in Bridgewater, New Jersey.

TDP’s source code repository resides at https://github.com/torbsd.

TDP is seeking funding to continue and extend its efforts. Please contact us if interested in assisting TDP, allowing us to dedicate more time to the project.

From the Attic

Attila blog post on OpenBSD Tor Browser Ports Status Update: July 2015, 4.5.3 (yes, I know it’s August)

Attila blog post on OpenBSD Tor Browser Ports Status Update: June 2015, v4.5.2

Attila blog post on OpenBSD Tor Browser Port Progress and Status

Attila blog post on Adventures in Ports: The Tor Browser

Early Rings “Porting Tor Browser to the BSDs” thread on Tor-BSD list


Copyright © 2016 by The Tor BSD Diversity Project (TDP). All Rights Reserved.

last updated: Fri Mar 17 19:25:51 2017 UTC