“Closing the gun-show loophole” has become a cause célèbre of American progressives.
As a European liberal living in the US, people expect me to march in lock-step with American progressives, and are surprised that I think this is a crock of shit.
Above all else, I am an empiricist and a realist.
The supposed purpose of “closing the gun show loophole” is to reduce the number of firearms falling into the hands of violent criminals, so it is reasonable to ask “where do criminals get their guns?” The best evidence we have is a 2002 report by the Department of Justice's Bureau of Justice Statistics (BJS) based on two surveys of inmates in state and federal prisons.
The upshot of this BJS report is that (roughly speaking) 40% of felons got their guns from friends and family, 40% got them from street or other illegal sources, and 20% purchased them from a variety of legal sources including gun shows where 0.6% and 0.7% got their guns. We don't know how many of the inmates surveyed would, at the time they acquired their guns, have passed a background check.
So, the long and the short of it is that “closing the gun show loophole”, at best, has the potential to reduce the number of firearms falling into the hands of criminals by 0.7%. All of the other sources, the sources where 99.3% of criminals get their guns, are still open.
In other words, the evidence indicates that “closing the gun show loophole” is utterly pointless.
Monday, December 7, 2015
Monday, November 16, 2015
What Google Can Learn from Amazon
Google are getting into retailing cell-phone service with Project Fi.
If you think about it, this is the first time Google has really gotten into retailing to the general public. Up to now, all of their services have been free to the consumer and paid for by business.
But it seems that, in their hubris, Google have decided to “wing it” in retail, rather than learning from the practices of people who've perfected it. Their first lesson is that the rules are different for a mom & pop store and a megacorp. In particular, where a small operation can often get away with an apology when they mess up; a megacorp can't.
The small operation gets away with an apology when a person in a genuine position of responsibility explains the extenuating circumstances and apologises to the customer with a modicum of sincerity. Even a bodega will usually make some kind of tangible gesture — a freebie, a sample, a discount, or something — by way of apology.
Logistically, a multi-billion dollar corporation cannot apologise in the same way, or the Board would have no time for anything but apologies. Consumers are also cynical of large corporations, and understand that the only meaningful way for a large corporation to express appreciation or contrition is with something of monetary value.
So a scripted pseudo-personal apology from a powerless support drone in Nevada is hollow and disingenuous, equivalent in sincerity to silently mouthing “I'm sorry” while giving the middle finger and winking.
The trick for a corporation is to give the disgruntled customer something that they value more highly than its cost to the company. Amazon understand this very well. They will give you a $5 or $10 credit or a month's free extension to your Prime subscription at the drop of a hat. You can't spend it anywhere else, you have to remain a customer to use it, they're guaranteed to get it back, they only pay cost for what you buy with it, and you'll probably buy something that you otherwise wouldn't that's worth more than the credit. Ultimately, it probably only costs them $1 to give you $5.
Google may be the Titans of Online Searching & Advertising, but they can still learn a thing or two from the Titans of Online Retail.
Google Project Fi(asco)
If you haven't heard, Google is getting into cell service with Project Fi(asco).
If the technology works, the fee structure has the potential to put the cat amongst the pigeons of the cell service oligopoly. I love it!
Like many Google projects going all the way back to Gmail, there's an “invite” stage before it's rolled out everyone, so I applied for, and got, an invitation. Yay!
But if you're thinking of doing the same, I'd say “don't bother”, for a number of reasons:
All 3 major cellphone service providers in the US are in the Customer Service Hall of Shame (positions 5, 7, & 8). Looks like Google wants to join them. What a pity.
If the technology works, the fee structure has the potential to put the cat amongst the pigeons of the cell service oligopoly. I love it!
Like many Google projects going all the way back to Gmail, there's an “invite” stage before it's rolled out everyone, so I applied for, and got, an invitation. Yay!
But if you're thinking of doing the same, I'd say “don't bother”, for a number of reasons:
- the sign-up process is misleading: your phone won't, in fact, leave the warehouse in the advertised 1-2 days, but in 5-6 weeks;
- it took filling out an online form followed by a 3-day farce of 11 emails to and from 4 different support personnel — Melissa, Dave the Would-Be Helper (marks for effort), Christina the Rude (or illiterate), and Tenisha — to arrive at the conclusion: cancelling the order;
- none of these people have the power to actually do anything unless you count writing chirpy corporate inanities and regurgitating FAQs;
- if my experience is representative, one in four won't even bother reading your email before “replying”; and
- at no time did they offer any kind of meaningful gesture of goodwill.
All 3 major cellphone service providers in the US are in the Customer Service Hall of Shame (positions 5, 7, & 8). Looks like Google wants to join them. What a pity.
Tuesday, November 3, 2015
Shortcomings of SFCU Online Banking
Like most people at Stanford, I bank with Stanford Federal Credit Union (SFCU).
Their lacklustre online banking system is fine for rudimentary day-to-day stuff, but it has a few dusty corners with some, frankly, ludicrous omissions and worryingly amateurish “features”.
Let's stick with tradition and talk about “Alice and Bob”, who are both SFCU customers.
I can think of three things off the top of my head...
Wrong.
Ironically, Bob can set up scheduled transfers to any account at any other bank using the ABA routing code and account number, but the external application/service SFCU use rejects SFCU's own ABA routing number. After some back-and-forth with SFCU customer service, their solution — to Bob wanting to set up a scheduled monthly transfer to Alice's account at the same branch of the same bank — was to mail Alice a monthly check.
Just thinking about it makes me smile. Was I wrong when I said “ludicrous”? Maybe I should've said “hilarious”.
Easy-peasy, right?
Wrong.
The “feature” simply doesn't work. After some back-and-forth with customer support, they tell Alice that she and Bob will just have to come in and fill out a paper form.
Here's how it should work:
Either support the feature or don't. It's not hard.
The online banking system has an integrated messaging system for customer support. It also has a timer that logs you out automatically after a period of time. I'm sure you can guess… Yes, it can try to log you out while you're typing. Apparently resetting the JavaScript timer that's already there on every keystroke was too much.
But that doesn't really concern me. Again, it's just an oversight, and because you are prompted before being automatically logged out, it's not a critical issue.
What gives me cause for concern is that the messaging system only allows alphanumeric characters plus a list of “allowed special characters”.
What this suggests is that the developers, quite rightly, feared:
If there's one thing I don't want to suspect was developed by dilettantes and acceptance tested by ignoramuses, it's the online banking system I use.
Their lacklustre online banking system is fine for rudimentary day-to-day stuff, but it has a few dusty corners with some, frankly, ludicrous omissions and worryingly amateurish “features”.
Let's stick with tradition and talk about “Alice and Bob”, who are both SFCU customers.
I can think of three things off the top of my head...
Shortcoming #1
Suppose Alice and Bob have some shared monthly expense that happens to be paid out of Alice's checking account. Bob wants to set up a scheduled transfer of, say, $100 to Alice's checking account on the 1st of every month. It was easy for Bob to do one-time transfers to Alice, and it was easy for Bob to set up scheduled transfers between his own savings and checking accounts, so putting the two together is doable, right?Wrong.
Ironically, Bob can set up scheduled transfers to any account at any other bank using the ABA routing code and account number, but the external application/service SFCU use rejects SFCU's own ABA routing number. After some back-and-forth with SFCU customer service, their solution — to Bob wanting to set up a scheduled monthly transfer to Alice's account at the same branch of the same bank — was to mail Alice a monthly check.
Just thinking about it makes me smile. Was I wrong when I said “ludicrous”? Maybe I should've said “hilarious”.
Shortcoming #2
Suppose Alice and Bob have a shared checking account. They want to open a second one. Bob clicks through to the screen for opening a new account, and there — much to his surprise and satisfaction — is a check-box that says “Share this account with Alice”. Strangely, Alice doesn't see this check-box when she tries, but Bob ticks the box and moves on.Easy-peasy, right?
Wrong.
The “feature” simply doesn't work. After some back-and-forth with customer support, they tell Alice that she and Bob will just have to come in and fill out a paper form.
Here's how it should work:
- The New Account page has a shared account option (via a tab, separate page, or whatever, the HCI/UI/UX details are unimportant to the current argument) with, inter alia, a text-box to enter the other customer's customer number
- Alice fills in Bob's customer number and clicks OK.
- Bob gets an email, clicks on a link, logs in, and it takes him to a page that says “Alice wants to open a shared checking account with you.”; it has 2 buttons “Accept” and “Decline”.
- If Bob accepts, the account is opened.
- If Bob declines, it isn't and Alice gets a message to that effect
Either support the feature or don't. It's not hard.
Shortcoming #3
The final shortcoming is simultaneously the most trivial and the most concerning. The previous two are oversights. Irritating, perhaps, but really just oversights or missing features.The online banking system has an integrated messaging system for customer support. It also has a timer that logs you out automatically after a period of time. I'm sure you can guess… Yes, it can try to log you out while you're typing. Apparently resetting the JavaScript timer that's already there on every keystroke was too much.
But that doesn't really concern me. Again, it's just an oversight, and because you are prompted before being automatically logged out, it's not a critical issue.
What gives me cause for concern is that the messaging system only allows alphanumeric characters plus a list of “allowed special characters”.
What this suggests is that the developers, quite rightly, feared:
- a SQL injection attack; and/or
- HTML special characters — like “<”, “>”, and “&” — being entered by the client, and later interpreted by the customer service agent's browser, such that the message entered by the client was not faithfully presented to the customer service agent.
If there's one thing I don't want to suspect was developed by dilettantes and acceptance tested by ignoramuses, it's the online banking system I use.
Friday, June 19, 2015
Using a C++ Flex Lexer with a C++ Bison Parser
I recently found myself revisiting lexing and parsing as part of my research. It's one of those cases where I would get away with ad-hoc parsing with line-splitting and regular expression matching, but the canonical alternative might ultimately turn out to be worth some extra effort for a number of reasons.
Many years ago, I wrote a SQL DDL parser as part of some object-relational mapping research into a “mutual containment” object model for transparently representing junction tables in relational databases. Even if I do say so myself, it was a neat idea. I don't know if anyone else has since thought of it independently and implemented it.
So, I decided to revisit flex and bison, the most popular versions of the venerable and classic compiler construction tools, lex and yacc. This time around, though, I anticipated a possible need for two parser/lexer subsystems, so I was interested in the C++ capabilities of both tools, since the “vanilla” C code they emit uses global variables.
The GNU Bison Manual has A Complete C++ Example that, unfortunately, rather narrowly interprets what it means to be “complete C++ example” to mean “an example where the bison bits are in C++”, and uses the vanilla flex lexer with global variables.
I found a few examples of using flex and bison with C++, but even the best of them only address one or the other, go off on irrelevant tangents, are short on explanation, contain outright misleading comments, use deprecated constructs, or all of the above.
All I wanted, was the minimal example of how to use a C++ flex lexer with a C++ bison parser. Some kind of “addendum” to the “complete” C++ example from the bison manual would be perfect!
*Crickets*.
So now, ladies and gentlemen, for your enjoyment, I have added to my Bitbucket “miscellany”, such an elucidation of Modifying the Bison “Complete C++ Example” to Use a C++ Flex Lexer as I formerly desired.
It's not as easy as it sounds.
Many years ago, I wrote a SQL DDL parser as part of some object-relational mapping research into a “mutual containment” object model for transparently representing junction tables in relational databases. Even if I do say so myself, it was a neat idea. I don't know if anyone else has since thought of it independently and implemented it.
So, I decided to revisit flex and bison, the most popular versions of the venerable and classic compiler construction tools, lex and yacc. This time around, though, I anticipated a possible need for two parser/lexer subsystems, so I was interested in the C++ capabilities of both tools, since the “vanilla” C code they emit uses global variables.
The GNU Bison Manual has A Complete C++ Example that, unfortunately, rather narrowly interprets what it means to be “complete C++ example” to mean “an example where the bison bits are in C++”, and uses the vanilla flex lexer with global variables.
I found a few examples of using flex and bison with C++, but even the best of them only address one or the other, go off on irrelevant tangents, are short on explanation, contain outright misleading comments, use deprecated constructs, or all of the above.
All I wanted, was the minimal example of how to use a C++ flex lexer with a C++ bison parser. Some kind of “addendum” to the “complete” C++ example from the bison manual would be perfect!
*Crickets*.
So now, ladies and gentlemen, for your enjoyment, I have added to my Bitbucket “miscellany”, such an elucidation of Modifying the Bison “Complete C++ Example” to Use a C++ Flex Lexer as I formerly desired.
It's not as easy as it sounds.
Tuesday, October 7, 2014
I'm a Wingnut and a Moonbat: Part 2
In an earlier post, I explained why I might be considered a “wingnut”. Briefly: I don't support the California assault weapons ban because it restricts access to certain firearms based on their cosmetic appearance rather than their functionality. Now I have to explain why I'm a “moonbat”.
When I say that the difference between the murder and assault rates in the US and the analogous rates in Western European countries can be explained almost entirely by economic inequality, I get accused of being a socialist, which I am, although not in the American sense of the word (where socialism is conflated with communism, totalitarianism, and oppression).
The evidence, however, is clear: about three quarters of the variance in murder and assault rates is explained by income inequality. This means that if you want to reduce violent crime in the US, you can address at most one quarter of the problem with all other measures combined: neither harsher sentences, favored by the right, nor gun control, favored by the left, are nearly as strongly correlated with violent crime as income inequality. Neither gun control nor prison sentences are statistically correlated with violent crime rates to a significant degree.
So, if you actually want to address violent crime, the elephant in the room is that you must address poverty, and not just absolute poverty, but relative poverty. To do that, you need to ensure that the distribution of wealth between rich and poor favors the poor more than it has in the past.
What policy changes ought to be introduced to ensure that the poor get a greater slice of the pie, and the rich a smaller slice, is open to debate, but any way you cut it, this sounds far too much like redistribution of wealth for any right-winger to resist the temptation to call me a “moonbat”.
When I say that the difference between the murder and assault rates in the US and the analogous rates in Western European countries can be explained almost entirely by economic inequality, I get accused of being a socialist, which I am, although not in the American sense of the word (where socialism is conflated with communism, totalitarianism, and oppression).
The evidence, however, is clear: about three quarters of the variance in murder and assault rates is explained by income inequality. This means that if you want to reduce violent crime in the US, you can address at most one quarter of the problem with all other measures combined: neither harsher sentences, favored by the right, nor gun control, favored by the left, are nearly as strongly correlated with violent crime as income inequality. Neither gun control nor prison sentences are statistically correlated with violent crime rates to a significant degree.
So, if you actually want to address violent crime, the elephant in the room is that you must address poverty, and not just absolute poverty, but relative poverty. To do that, you need to ensure that the distribution of wealth between rich and poor favors the poor more than it has in the past.
What policy changes ought to be introduced to ensure that the poor get a greater slice of the pie, and the rich a smaller slice, is open to debate, but any way you cut it, this sounds far too much like redistribution of wealth for any right-winger to resist the temptation to call me a “moonbat”.
Friday, October 3, 2014
I'm a Wingnut and a Moonbat: Part 1
For those unfamiliar with American political discourse, a “wingnut”, so the stereotype goes, is a gun-totin' redneck Evangelical Christian Republican-voting moron who gets all of his/her opinions straight from Rush Limbaugh. A “moonbat”, by contrast, is a gun-grabbin' city-dwelling atheist Democrat-voting commie who gets all of his/her unconsidered opinions straight from The Daily Show.
Now, you might think that an advocate of evidence-based policy might end up somewhere in the middle and would be neither a wingnut nor a moonbat, but it turns out that such a person is actually both.
Here's one example (I'll address the other side in Part 2)...
California has, arguably, the strictest gun control of any state. I come from a place where firearm licensing is immeasurably stricter, basically the strictest in the world. There is no legal provision for a license for a handgun or a center-fire rifle. You can get a license for a shotgun, double-barreled or pump with a 3 round magazine, or a .22 rim-fire rifle, but that's it and it's not easy. Broadly speaking, as a people, we have no tradition of firearms ownership and we neither need nor want firearms. Our police are mostly unarmed, and we like it that way.
As a person living in the US on a visa, I can't legallyown buy (see edit note, below) a firearm. I find them interesting, of course, as any kid who played cops & robbers patterned after American TV shows might, but basically, I have no dog in this fight. I'm undecided whether, if I were allowed, I would end up having a ridiculous arsenal of firearms just because they're cool and fun and I'm a big kid, or I just wouldn't bother because they're dangerous, expensive, and the likelihood of ever actually needing to use one in the wealthy suburban part of California where I live is essentially zero.
But here's the thing. I read California's so-called “assault weapons” ban, and it is, by any objective standard, absurd. It is absurd, not because of its abuse of terminology (it actually bans some facsimiles of assault rifles), but because it regulates the appearance of the firearm, not any feature of its operation. In short, it regulates only the cosmetic appearance of the firearm. Don't believe me?
OK...
This is illegal to buy, sell, import, etc. in California (legal to own if grandfathered in). You will probably be charged with a felony (technically, it's a “wobbler”) if you are caught with one of these and can't prove that you bought it before the ban:
This functionally identical rifle — firing identical rounds with an identical mechanism from an identical magazine down an identical barrel at an identical rate of one per trigger pull — is legal to buy and own in California (assuming in both cases that the magazine holds no more than 10 rounds, despite looking like 20 or 30 round magazines):
What makes the legal difference?
The pistol-style grip above is illegal; the one below, fused to the stock, is legal. That's the difference. And don't think that federal laws are much better. If you add a handle-like grip to the barrel of either of the above without permission from the ATF and a $200 tax, that's a felony.
This, we are supposed to believe, is what is going to protect children from a Newtown-style shooting: making guns have the right kind of handle and making it a felony to have the wrong kind of handle or, perish the thought, an extra handle.
So now you know why I agree with the gun rights activists and say “the California Assault Weapons Ban is one of the most astonishingly stupid things I have ever heard of”, but only wingnuts say things like that, so I must be a wingnut, right?
Edit: in the original version of this post, I said that (as someone here on a visa) I couldn't legally own a firearm. That's probably not exactly true. 18 USC 922 (d)(5)(B) criminalizes the sale, or other transfer, of a firearm or ammunition to a person “who, being an alien— [...] has been admitted to the United States under a nonimmigrant visa [...]” (that's me). Not that I'd realistically consider doing it, but it might be perfectly legal, at least under federal law, for me to make a firearm (e.g. from an 80% complete lower) and ammunition (e.g. by reloading), since the power that the federal government uses for 18 USC 922 derives from the Commerce Clause and so, strictly speaking, regulates only interstate or international trade. There's even an argument that I could legally purchase ammunition manufactured in California (since it would not involve interstate or international commerce and, therefore, cannot fall under the authority granted by the Commerce Clause). In any case, it's a minor technicality, there's probably some other law that prohibits it at the federal or state level, and I'm not so enamoured with the idea of firearms ownership that I'd bother risking it. Nevertheless, I've changed “own” to “buy” since, as far as I know, that's more accurate.
Now, you might think that an advocate of evidence-based policy might end up somewhere in the middle and would be neither a wingnut nor a moonbat, but it turns out that such a person is actually both.
Here's one example (I'll address the other side in Part 2)...
California has, arguably, the strictest gun control of any state. I come from a place where firearm licensing is immeasurably stricter, basically the strictest in the world. There is no legal provision for a license for a handgun or a center-fire rifle. You can get a license for a shotgun, double-barreled or pump with a 3 round magazine, or a .22 rim-fire rifle, but that's it and it's not easy. Broadly speaking, as a people, we have no tradition of firearms ownership and we neither need nor want firearms. Our police are mostly unarmed, and we like it that way.
As a person living in the US on a visa, I can't legally
But here's the thing. I read California's so-called “assault weapons” ban, and it is, by any objective standard, absurd. It is absurd, not because of its abuse of terminology (it actually bans some facsimiles of assault rifles), but because it regulates the appearance of the firearm, not any feature of its operation. In short, it regulates only the cosmetic appearance of the firearm. Don't believe me?
OK...
This is illegal to buy, sell, import, etc. in California (legal to own if grandfathered in). You will probably be charged with a felony (technically, it's a “wobbler”) if you are caught with one of these and can't prove that you bought it before the ban:
 |
| Illegal “Assault Weapon” in California |
 |
| Legal “Sporting Rifle” in California |
The pistol-style grip above is illegal; the one below, fused to the stock, is legal. That's the difference. And don't think that federal laws are much better. If you add a handle-like grip to the barrel of either of the above without permission from the ATF and a $200 tax, that's a felony.
This, we are supposed to believe, is what is going to protect children from a Newtown-style shooting: making guns have the right kind of handle and making it a felony to have the wrong kind of handle or, perish the thought, an extra handle.
Wow!
So now you know why I agree with the gun rights activists and say “the California Assault Weapons Ban is one of the most astonishingly stupid things I have ever heard of”, but only wingnuts say things like that, so I must be a wingnut, right?
Edit: in the original version of this post, I said that (as someone here on a visa) I couldn't legally own a firearm. That's probably not exactly true. 18 USC 922 (d)(5)(B) criminalizes the sale, or other transfer, of a firearm or ammunition to a person “who, being an alien— [...] has been admitted to the United States under a nonimmigrant visa [...]” (that's me). Not that I'd realistically consider doing it, but it might be perfectly legal, at least under federal law, for me to make a firearm (e.g. from an 80% complete lower) and ammunition (e.g. by reloading), since the power that the federal government uses for 18 USC 922 derives from the Commerce Clause and so, strictly speaking, regulates only interstate or international trade. There's even an argument that I could legally purchase ammunition manufactured in California (since it would not involve interstate or international commerce and, therefore, cannot fall under the authority granted by the Commerce Clause). In any case, it's a minor technicality, there's probably some other law that prohibits it at the federal or state level, and I'm not so enamoured with the idea of firearms ownership that I'd bother risking it. Nevertheless, I've changed “own” to “buy” since, as far as I know, that's more accurate.
Monday, August 25, 2014
The Problem with Fracking: Part 3
In the first and second parts of this series, I argued that aquifer contamination from the completion in a fracked gas well is highly improbable, and that fracking fluid, as distinct from waste water, is benign.
In this part, which could easily be titled “When Fracktivism Isn't About Fracking”, I take a look at “fraccidents” in Pennsylvania.
Pennsylvania has been a gas-producing state since the mid 19th century, but has seen a boom in gas production since fracking enabled the exploitation of the Marcellus shale. There are estimated to be 350,000 oil and gas wells in Pennsylvania, the vast majority of which are old, inactive wells. EIA data shows a sevenfold increase in gas production in the four years after horizontal drilling (more-or-less synonymous with fracking) began in earnest around 2008. Currently, there are about 9,000 active wells in Pennsylvania, with a new well being drilled every day (on average). The following EIA animation shows wells drilled in Pennsylvania between January 2005 and April 2012:
All in all, I figure that the Earth Justice “fraccidents” represent 45 separate events.
But how many of these are actually to do with fracking?
Let's have a look at a few of these fraccidents...
Sticking out like a sore thumb, there's a lonely skull & crossbones about 30 miles south of Harrisburg, and over 100 miles east of any well (fracked or otherwise) known to EIA. It turns out that this is actually a drilling mud spill that happened when a gas pipeline was being drilled 13 feet under a creek. This might be related to the Pennsylvania gas industry, but has nothing whatsoever to do with fracking and, even if it did, it's a very minor event. Equally, representing a spill of 2-3 cubic yards of bentonite (a kind of clay) with a skull and crossbones reeks of alarmism.
More appropriate use of a “skull & crossbones” is in the southwest corner (as it happens, the event actually happened in West Virginia, but appears in Pennsylvania, presumably due to some innocent error), where 10,000 fish were killed over a 30-mile stretch of river. What caused this? An algal bloom caused by discharges from coal-mining. Nothing to do with fracking, or even the oil & gas industry.
There's a water-contamination event in Hickory (just west of Pittsburgh) in December 2005. The problem is that the first horizontal well in this area was recorded by EIA in September 2007. So this event, if it was was due to a gas well at all, was due to a conventional well and, again, nothing to do with fracking.
There are several examples of compressors catching fire, a faulty tailgate on a truck allowing drill cuttings to fall out onto the road, a truck leaking hydrochloric acid, drilling mud spills, diesel spills, or methane leaks. These are nothing whatsoever to do with hydraulic fracturing: they might just as well have happened with any kind of gas well, fracked or not, or in another industry entirely.
In fairness, there are several events that can plausibly be linked to fracking in some way. In most cases, these are either actual wastewater leaks or events probably caused by wastewater leaks. For example, there are several cases of high levels of metals, including arsenic, and aromatics, including benzene, being found in soil or drinking water. Although the causal connection with fracking is tenuous, they are most likely attributable to wastewater leaks, and the wastewater is from frack-jobs. There have been a number of casing and cementing failures that probably wouldn't have happened in a conventional well, probably happening due to the extreme pressure used during fracking.
So, how many of the 45 “fraccidents” survive cursory fact-checking?
Being generous? About half. The other half range from outright lies (fish-kills due to coal-mining) to gross exaggeration (cuttings falling from a truck) to falsely attributing accidents to fracking when they're just associated with the gas industry generally (compressor accidents, pipeline leaks).
So is, say, 25 real “fraccidents” too many? Maybe. The reality is that oil & gas extraction is dangerous. All extractive industries carry the potential for pollution, whether that's fracking for shale gas or mining neodymium for the permanent magnets in wind turbines. With 9,000 active wells in Pennsylvania producing 4 trillion cubic feet of gas over 6 years, 25 accidents doesn't seem like a whole lot to me.
It isn't a great surprise that much fracktivism is disingenuous. After all, the movie that essentially started the whole anti-fracking hysteria, Gasland (on Netflix), is a stunningly dishonest piece, as exposed by FrackNation (also on Netflix). The truth is that almost all real “fraccidents” are wastewater leaks, which are actually pretty rare, but do happen occasionally.
The idea that we need better regulation of wastewater storage and disposal practices is worthy of consideration, and enforcement must be adequately resourced, but the way to argue these points is not with alarmist lies, hysterical exaggeration, and presenting personal anecdotes as fact.
Josh Fox (Gasland) found a dozen people who claim that “fracking” has destroyed their lives. Phelim McAleer (FrackNation) found a dozen who claim that “fracking” is essential to their economic survival. Neither is a sound basis for forming a personal opinion, much less government policy. I could easily find a dozen people who claim to have been abducted and anally probed by aliens. I could make an engaging and emotional documentary about it. But that doesn't mean that we need a government policy to deal with the alien abduction problem. Policy should be based on expert analysis of evidence.
In this part, which could easily be titled “When Fracktivism Isn't About Fracking”, I take a look at “fraccidents” in Pennsylvania.
Pennsylvania has been a gas-producing state since the mid 19th century, but has seen a boom in gas production since fracking enabled the exploitation of the Marcellus shale. There are estimated to be 350,000 oil and gas wells in Pennsylvania, the vast majority of which are old, inactive wells. EIA data shows a sevenfold increase in gas production in the four years after horizontal drilling (more-or-less synonymous with fracking) began in earnest around 2008. Currently, there are about 9,000 active wells in Pennsylvania, with a new well being drilled every day (on average). The following EIA animation shows wells drilled in Pennsylvania between January 2005 and April 2012:
As it happens, this is more-or-less the same time span covered by Earth Justice's record of “fraccidents” in Google Maps. In mid-late August, 2014 (the time of writing of this blog entry), they place 38 “skull and crossbones” symbols over Pennsylvania for what are, presumably, poisoning events of some kind resulting in death or serious injury; at least a major fish-kill or something, right? You don't use a skull and crossbones to indicate something like rocks falling from the back of a truck with a faulty tailgate, after all, right? It turns out that you do.
 |
| “Fraccidents” in Pennsylvania |
When you click on a skull & crossbones, you sometimes find that it corresponds to two or three separate events that might be revealed at a higher zoom level (or not; sometimes they are in the associated text or linked articles). It can sometimes be hard to disentangle where and when, exactly, a particular event occurred since, often, linked articles make reference to events that happened elsewhere in Pennsylvania at some indeterminate time in the past (the Zimmerman vs. Atlas Energy lawsuit is mentioned several times). Equally, there are occasions when an opportunity to include a separate skull and crossbones has been missed.
 |
| “Fraccidents” Overlaid on EIA Well Data |
But how many of these are actually to do with fracking?
Let's have a look at a few of these fraccidents...
Sticking out like a sore thumb, there's a lonely skull & crossbones about 30 miles south of Harrisburg, and over 100 miles east of any well (fracked or otherwise) known to EIA. It turns out that this is actually a drilling mud spill that happened when a gas pipeline was being drilled 13 feet under a creek. This might be related to the Pennsylvania gas industry, but has nothing whatsoever to do with fracking and, even if it did, it's a very minor event. Equally, representing a spill of 2-3 cubic yards of bentonite (a kind of clay) with a skull and crossbones reeks of alarmism.
More appropriate use of a “skull & crossbones” is in the southwest corner (as it happens, the event actually happened in West Virginia, but appears in Pennsylvania, presumably due to some innocent error), where 10,000 fish were killed over a 30-mile stretch of river. What caused this? An algal bloom caused by discharges from coal-mining. Nothing to do with fracking, or even the oil & gas industry.
There's a water-contamination event in Hickory (just west of Pittsburgh) in December 2005. The problem is that the first horizontal well in this area was recorded by EIA in September 2007. So this event, if it was was due to a gas well at all, was due to a conventional well and, again, nothing to do with fracking.
There are several examples of compressors catching fire, a faulty tailgate on a truck allowing drill cuttings to fall out onto the road, a truck leaking hydrochloric acid, drilling mud spills, diesel spills, or methane leaks. These are nothing whatsoever to do with hydraulic fracturing: they might just as well have happened with any kind of gas well, fracked or not, or in another industry entirely.
In fairness, there are several events that can plausibly be linked to fracking in some way. In most cases, these are either actual wastewater leaks or events probably caused by wastewater leaks. For example, there are several cases of high levels of metals, including arsenic, and aromatics, including benzene, being found in soil or drinking water. Although the causal connection with fracking is tenuous, they are most likely attributable to wastewater leaks, and the wastewater is from frack-jobs. There have been a number of casing and cementing failures that probably wouldn't have happened in a conventional well, probably happening due to the extreme pressure used during fracking.
So, how many of the 45 “fraccidents” survive cursory fact-checking?
Being generous? About half. The other half range from outright lies (fish-kills due to coal-mining) to gross exaggeration (cuttings falling from a truck) to falsely attributing accidents to fracking when they're just associated with the gas industry generally (compressor accidents, pipeline leaks).
So is, say, 25 real “fraccidents” too many? Maybe. The reality is that oil & gas extraction is dangerous. All extractive industries carry the potential for pollution, whether that's fracking for shale gas or mining neodymium for the permanent magnets in wind turbines. With 9,000 active wells in Pennsylvania producing 4 trillion cubic feet of gas over 6 years, 25 accidents doesn't seem like a whole lot to me.
It isn't a great surprise that much fracktivism is disingenuous. After all, the movie that essentially started the whole anti-fracking hysteria, Gasland (on Netflix), is a stunningly dishonest piece, as exposed by FrackNation (also on Netflix). The truth is that almost all real “fraccidents” are wastewater leaks, which are actually pretty rare, but do happen occasionally.
The idea that we need better regulation of wastewater storage and disposal practices is worthy of consideration, and enforcement must be adequately resourced, but the way to argue these points is not with alarmist lies, hysterical exaggeration, and presenting personal anecdotes as fact.
Josh Fox (Gasland) found a dozen people who claim that “fracking” has destroyed their lives. Phelim McAleer (FrackNation) found a dozen who claim that “fracking” is essential to their economic survival. Neither is a sound basis for forming a personal opinion, much less government policy. I could easily find a dozen people who claim to have been abducted and anally probed by aliens. I could make an engaging and emotional documentary about it. But that doesn't mean that we need a government policy to deal with the alien abduction problem. Policy should be based on expert analysis of evidence.
Wednesday, August 13, 2014
The Problem with Fracking: Part 2
In the first part of this series, I demonstrated, by correcting a representative anti-fracking graphic with a few simple facts, that contamination of aquifers by fracking fluid or hydrocarbons from the completion of a shale gas well is highly implausible due to the natural barrier between the two in the Marcellus shale.
In this second part, I will argue both that the composition of fracking fluid is benign and that, even if it weren't, it wouldn't matter. Hopefully putting to bed another fracktivist fable.
Much fracktivist propaganda exploits public scientific ignorance and chemophobia (“Eek! It's full of chemicals!”).
Not everyone is, or can be, highly scientifically literate, but to most of us, who don't know our isomethyloxidocarbonoxybenzoic acid from our isopropanylmethylphosphonofluoridate, these claims about an absolutely real fracking fluid component sound very scary:
The most common metric of toxicity is the median fatal dose, (LD50) usually expressed in wt/wt form, such as mg/kg (milligrams of toxin per kg of bodyweight) for lab animals like rats or rabbits (with certain caveats, we usually make the assumption that this “scales up” to people, at least roughly). The LD50 for water is 90 g/kg; this means that if 100 people, each weighing 80 kg (176 lbs) drank 7.2 liters (1.9 US gallons) of water in one sitting, half of them would die. Water is toxic.
This toxin, water, is “a powerful industrial solvent”. So what! Alternative uses of a substance tell you absolutely nothing about its safety.
Both sides of the fracking debate are absolutely rife with “danger by analogy” or “safety by analogy” fallacies, sometimes about the same substance. For example, a fracktivist might tell you that potassium chloride is the poison used in death by lethal injection, while a fracking proponent might tell you that potassium chloride is a harmless substitute for regular table salt. Both of these statements, on their own, are entirely true.
Now it would be tempting to damn both sides for making such fallacious arguments. After all, both “true” statements in the preceding paragraph are equally misleading, aren't they? Well, no, not quite. When we're talking about fracking, the concern is not that someone will accidentally inject fracking fluid intravenously, it's that someone will inadvertently ingest something contaminated with fracking fluid. In context, one of the analogies parallels the implied concern (ingestion vs. ingestion) and the other one doesn't (ingestion vs. injection).
Ultimately, there are three key points here:
Now that you want to lynch the parent from the nearest tree for adding dangerous chemicals to their child's food, let me ask you a second question:
Whenever you see anti-fracking propaganda saying that “oil companies used millions of gallons of products containing benzene and toluene in fracking fluid”, as in the above-linked article, it's undeniably true. Light hydrocarbons — including benzene, toluene, and others — occur incidentally in minute quantities in petroleum distillates, which are used directly as lubricants and indirectly as carriers for other additives. They are present, just as acrylamide is present in baked goods, such as a hot dog bun, and nitrosamines are present in cured and cooked meats, such as a wiener. But there is a huge difference between something being present incidentally in minute quantities, and being added with deliberate intent and depraved indifference to public safety.
The truth is that “Big Oil” is no more adding benzene to fracking fluid than Oscar Mayer is adding nitrosamines to bacon.
Natural gas, including shale gas, is a mixture of a large number of hydrocarbons. The composition varies widely from reservoir to reservoir, just as the composition of oil from Saudi Arabia, Texas, and the North Sea differ. We can say that it is mostly methane, ethane, propane, and butane, but also that it contains, amongst other things, benzene, toluene, xylene, and other aromatics.
The lithostatic overburden pressure (due to the rock above) in the Marcellus formation is probably something in the region of 400 to 500 times atmospheric pressure and the temperature is also considerably higher than at the surface. When the reservoir fluid is brought to the surface, it expands and cools. Some constituents remain or become natural gas and some condense into a liquid, called condensate. It can be pretty difficult to get an idea of the exact composition “deep in the ground” because data are published separately for natural gas and condensates at surface conditions, but the gas typically contains only minute traces of aromatics (0.005%), while the liquid condensate can contain aromatics at concentrations between 0.2 and 12%. What we can say is that there are a lot of aromatics in the ground and the concentration is non-negligible.
Why would a reasonable person care about pumping fracking fluid that has a trace of toluene in it into a formation where the concentration of toluene is 100 times higher? It makes no sense. We would all agree that discarding a lit cigarette butt is generally bad because it creates a fire hazard, but discarding it into a blast furnace is another thing entirely.
We can vacillate endlessly about individual components, but ultimately, the simple answer is “no”.
Furthermore, whatever the components of any real-life fracking fluid, the same in higher concentrations or much, much worse is already down there, and some of it ends up in the fracking fluid that returns to the surface when the frack-job is done. This is the reason why it is the waste water after fracking that should be the focus of environmental concern, and not what “Big Oil” is putting in the fracking fluid. Conflating the two, as fracktivists are wont to do, is worse again.
Saying things like “fracking is pumping millions of gallons of a cocktail of poisons into the ground” is a shibboleth for reflexive opposition and ignorance of the real environmental issues.
In the next instalment, I will deal with the question: “When is fracktivism really about fracking?”
In this second part, I will argue both that the composition of fracking fluid is benign and that, even if it weren't, it wouldn't matter. Hopefully putting to bed another fracktivist fable.
Sensationalism and Ignorance
The myth goes something like this: “fracking is pumping millions of gallons of a cocktail of poisons into the ground”. The majority of people who believe this do so because they've been misinformed, usually by people who claim to understand fracking and ought to know better, often by people who actually do know better and are compartmentalizing, and rarely by idealogical liars.Much fracktivist propaganda exploits public scientific ignorance and chemophobia (“Eek! It's full of chemicals!”).
Not everyone is, or can be, highly scientifically literate, but to most of us, who don't know our isomethyloxidocarbonoxybenzoic acid from our isopropanylmethylphosphonofluoridate, these claims about an absolutely real fracking fluid component sound very scary:
- Component X is a powerful industrial solvent.
- Component X is based on the highly reactive hydroxyl radical, known to mutate DNA, denature proteins, disrupt cell membranes, and chemically alter critical neurotransmitters.
- Component X is a CNS (central nervous system) depressant, and can cause cerebral edema and brain herniation.
- High levels of Component X have been found in tumors excised from cancer patients.
The most common metric of toxicity is the median fatal dose, (LD50) usually expressed in wt/wt form, such as mg/kg (milligrams of toxin per kg of bodyweight) for lab animals like rats or rabbits (with certain caveats, we usually make the assumption that this “scales up” to people, at least roughly). The LD50 for water is 90 g/kg; this means that if 100 people, each weighing 80 kg (176 lbs) drank 7.2 liters (1.9 US gallons) of water in one sitting, half of them would die. Water is toxic.
This toxin, water, is “a powerful industrial solvent”. So what! Alternative uses of a substance tell you absolutely nothing about its safety.
Both sides of the fracking debate are absolutely rife with “danger by analogy” or “safety by analogy” fallacies, sometimes about the same substance. For example, a fracktivist might tell you that potassium chloride is the poison used in death by lethal injection, while a fracking proponent might tell you that potassium chloride is a harmless substitute for regular table salt. Both of these statements, on their own, are entirely true.
Now it would be tempting to damn both sides for making such fallacious arguments. After all, both “true” statements in the preceding paragraph are equally misleading, aren't they? Well, no, not quite. When we're talking about fracking, the concern is not that someone will accidentally inject fracking fluid intravenously, it's that someone will inadvertently ingest something contaminated with fracking fluid. In context, one of the analogies parallels the implied concern (ingestion vs. ingestion) and the other one doesn't (ingestion vs. injection).
Ultimately, there are three key points here:
- misleading phraseology can make anything look extremely dangerous to a superficial reading;
- simply asserting that something is “toxic” is meaningless without information about dosage (how much) and delivery (oral, skin, IV);
- also, using more specific terms like “neurotoxic”, “hepatotoxic”, “cytotoxic”, “carcinogenic”, “teratogenic”, “mutagenic” doesn't change that; and
- alternative uses for a substance are not indicative of their safety or otherwise.
Incidental Constituents
Let me ask a question:- What would you think of a father who added acrylamide and nitrosamines to his child's food?
Now that you want to lynch the parent from the nearest tree for adding dangerous chemicals to their child's food, let me ask you a second question:
- What would you think of a father who gave his child a hot dog?
Whenever you see anti-fracking propaganda saying that “oil companies used millions of gallons of products containing benzene and toluene in fracking fluid”, as in the above-linked article, it's undeniably true. Light hydrocarbons — including benzene, toluene, and others — occur incidentally in minute quantities in petroleum distillates, which are used directly as lubricants and indirectly as carriers for other additives. They are present, just as acrylamide is present in baked goods, such as a hot dog bun, and nitrosamines are present in cured and cooked meats, such as a wiener. But there is a huge difference between something being present incidentally in minute quantities, and being added with deliberate intent and depraved indifference to public safety.
The truth is that “Big Oil” is no more adding benzene to fracking fluid than Oscar Mayer is adding nitrosamines to bacon.
The Same, or Worse
Some of the most hysterical reactions to fracking fluid constituents are prompted by aromatics — hydrocarbons that contain 6 carbon atoms in a loop called a benzene ring that are often carcinogenic — such as benzene itself, toluene (a benzene ring with one methyl group), and xylene (a benzene ring with two methyl groups).Natural gas, including shale gas, is a mixture of a large number of hydrocarbons. The composition varies widely from reservoir to reservoir, just as the composition of oil from Saudi Arabia, Texas, and the North Sea differ. We can say that it is mostly methane, ethane, propane, and butane, but also that it contains, amongst other things, benzene, toluene, xylene, and other aromatics.
The lithostatic overburden pressure (due to the rock above) in the Marcellus formation is probably something in the region of 400 to 500 times atmospheric pressure and the temperature is also considerably higher than at the surface. When the reservoir fluid is brought to the surface, it expands and cools. Some constituents remain or become natural gas and some condense into a liquid, called condensate. It can be pretty difficult to get an idea of the exact composition “deep in the ground” because data are published separately for natural gas and condensates at surface conditions, but the gas typically contains only minute traces of aromatics (0.005%), while the liquid condensate can contain aromatics at concentrations between 0.2 and 12%. What we can say is that there are a lot of aromatics in the ground and the concentration is non-negligible.
Why would a reasonable person care about pumping fracking fluid that has a trace of toluene in it into a formation where the concentration of toluene is 100 times higher? It makes no sense. We would all agree that discarding a lit cigarette butt is generally bad because it creates a fire hazard, but discarding it into a blast furnace is another thing entirely.
The Real Question
The proper concern about any component of fracking fluid is “is this substance harmful in the concentrations found in fracking fluid?”We can vacillate endlessly about individual components, but ultimately, the simple answer is “no”.
Furthermore, whatever the components of any real-life fracking fluid, the same in higher concentrations or much, much worse is already down there, and some of it ends up in the fracking fluid that returns to the surface when the frack-job is done. This is the reason why it is the waste water after fracking that should be the focus of environmental concern, and not what “Big Oil” is putting in the fracking fluid. Conflating the two, as fracktivists are wont to do, is worse again.
Saying things like “fracking is pumping millions of gallons of a cocktail of poisons into the ground” is a shibboleth for reflexive opposition and ignorance of the real environmental issues.
In the next instalment, I will deal with the question: “When is fracktivism really about fracking?”
Monday, August 11, 2014
The Problem with Fracking: Part 1
The problem with fracking is that fracking itself is fairly benign. It's really the associated wastewater disposal that presents the most serious environmental challenge.
Almost every day, I see anti-fracking posts on Facebook. About once a month, I get handed anti-fracking leaflets on the street. All of these tell me that fracking is bad because of, amongst other things, the risk of contamination of drinking water aquifers with fracking fluid and hydrocarbons from the completion, as depicted in this graphic that I found via Google Image Search:
The problem with this is that it's a lie. In this first instalment of The Problem with Fracking series of blog posts, I'll explain why.
Aquifers used for drinking water and irrigation are usually shallow, something like a few hundred feet down. Anything over 500 feet down is considered a “deep bedrock aquifer”. There are a couple of reasons why shallow aquifers are preferred: economics and water quality. As a general rule, the deeper you go, the worse the water. The reason for this is that temperature and pressure rise as you go down, with a corresponding increase in the proportion of dissolved salts in the water. Geothermal plants, where the water comes from deep underground at high temperature, regularly measure salinities as high as 20% (sea water is about 3.5%).
Most shale gas plays occur at depths of around 10,000 feet. The famous Marcellus shale is often chosen to exemplify the problems with fracking because it is well-known, vast, and unusually shallow in places, but still about 5,000 feet to the top of the formation, with an average well depth of 6,300 feet.
I've said that the graphic above is a lie, so it's reasonable to ask what happens if we correct it? I took the number of pixels from the “surface” to the bottom of the water well on the right, dramatically shown with a big yellow flame on top, and found it to be 90 pixels. The average depth of a domestic water well in Pennsylvania is 200 feet, or 4.4 feet per pixel. At this vertical scale, the picture becomes:
So as not to be accused of hypocrisy, here's what it looks like if we correct the aquifer depth and fracture tops to their worst-case values (1,000 feet and 4,500 feet, respectively):
So, now we have something a little more honest, but there's also the issue of the composition of the rock. With its choice of color, the graphic implies that the gas play and the aquifer are part of the same rock formation. I have little doubt that the color is chosen to be suggestive of sandstone, which is typical of the permeable rock found in both aquifers and oil reservoirs, but these aren't called shale gas plays for nothing, and, in this case, it isn't called the Marcellus shale for nothing, so let's add somewhat plausible Pennsylvania stratigraphy (I've changed the color of the gas shale from black to light bluish gray to retain easy visibility of other features, such as the completion; other colors are unchanged):
This final diagram couldn't honestly be described as “representative”, because it stretches the facts to within a hair's breadth of breaking-point, yet it still leaves an impermeable barrier totalling over a kilometer of rock in three facies between the worst-case fractures and the worst-case aquifer. At least this diagram bears some passing resemblance to the truth, but the original is outright
It is possible for groundwater contamination to occur at the surface, but fear of aquifer contamination from the completion, as depicted in the original graphic, is about as reasonable as the premise of Black Sheep.
The next installment of this series will demonstrate why another hobby-horse of anti-fracking hysteria — the composition of fracking fluid — holds no water (pun intended).
Almost every day, I see anti-fracking posts on Facebook. About once a month, I get handed anti-fracking leaflets on the street. All of these tell me that fracking is bad because of, amongst other things, the risk of contamination of drinking water aquifers with fracking fluid and hydrocarbons from the completion, as depicted in this graphic that I found via Google Image Search:
 |
| “Gasland” Fracking Graphic |
The problem with this is that it's a lie. In this first instalment of The Problem with Fracking series of blog posts, I'll explain why.
Aquifers used for drinking water and irrigation are usually shallow, something like a few hundred feet down. Anything over 500 feet down is considered a “deep bedrock aquifer”. There are a couple of reasons why shallow aquifers are preferred: economics and water quality. As a general rule, the deeper you go, the worse the water. The reason for this is that temperature and pressure rise as you go down, with a corresponding increase in the proportion of dissolved salts in the water. Geothermal plants, where the water comes from deep underground at high temperature, regularly measure salinities as high as 20% (sea water is about 3.5%).
Most shale gas plays occur at depths of around 10,000 feet. The famous Marcellus shale is often chosen to exemplify the problems with fracking because it is well-known, vast, and unusually shallow in places, but still about 5,000 feet to the top of the formation, with an average well depth of 6,300 feet.
 |
| Marcellus Shale |
I've said that the graphic above is a lie, so it's reasonable to ask what happens if we correct it? I took the number of pixels from the “surface” to the bottom of the water well on the right, dramatically shown with a big yellow flame on top, and found it to be 90 pixels. The average depth of a domestic water well in Pennsylvania is 200 feet, or 4.4 feet per pixel. At this vertical scale, the picture becomes:
So as not to be accused of hypocrisy, here's what it looks like if we correct the aquifer depth and fracture tops to their worst-case values (1,000 feet and 4,500 feet, respectively):
So, now we have something a little more honest, but there's also the issue of the composition of the rock. With its choice of color, the graphic implies that the gas play and the aquifer are part of the same rock formation. I have little doubt that the color is chosen to be suggestive of sandstone, which is typical of the permeable rock found in both aquifers and oil reservoirs, but these aren't called shale gas plays for nothing, and, in this case, it isn't called the Marcellus shale for nothing, so let's add somewhat plausible Pennsylvania stratigraphy (I've changed the color of the gas shale from black to light bluish gray to retain easy visibility of other features, such as the completion; other colors are unchanged):
This final diagram couldn't honestly be described as “representative”, because it stretches the facts to within a hair's breadth of breaking-point, yet it still leaves an impermeable barrier totalling over a kilometer of rock in three facies between the worst-case fractures and the worst-case aquifer. At least this diagram bears some passing resemblance to the truth, but the original is outright
It is possible for groundwater contamination to occur at the surface, but fear of aquifer contamination from the completion, as depicted in the original graphic, is about as reasonable as the premise of Black Sheep.
The next installment of this series will demonstrate why another hobby-horse of anti-fracking hysteria — the composition of fracking fluid — holds no water (pun intended).
Thursday, August 7, 2014
On the Datatype of Matrix Indices
Computer science purists often try to convince me that size_t is the appropriate type for matrix and vector indices in C or C++. This is always asserted without further justification as if it were commonly accepted. When I say “a matrix index isn't the size of anything, so the burden of establishing that size_t is the ‘one true type’ for these indices lies with you”, I'm generally either insulted or berated, but a strong rational justification has yet to be presented.
My position is that any unsigned integral type is unsuitable for use as a general matrix or array index in computational science. My argument is essentially a utilitarian one: that pragmatism should win over naïve pedantry.
The most succinct expression of my reasoning is that, from an algorithmic perspective, it is, in general, highly desirable that indices approximate a group under addition (as closely as we can with finite size), and that using any unsigned integer type unnecessarily deprives index variables of the ability to store the additive inverse.
The argument that, formally, matrix indices are natural numbers, {1, 2, 3, ...}, holds very little weight in this argument, since C and C++ already base arrays at zero. I maintain that the ultimate reason for this is the utility of having the additive identity, 0 (zero).
Matrix-oriented languages, like Fortran and Matlab, despite basing indices at 1, do not restrict the datatype of variables used as indices strictly to the set of natural numbers. That is to say that, in these languages that are specifically and explicitly matrix- and vector-oriented, provided that an index variable is a valid index when it is used as an index, the values that it takes at other points in a program are unconstrained. I would take some convincing that the exact opposite should be the case in C or C++, yet this is precisely the position that purists ridicule me for opposing. If you were to suggest to a Fortran or Matlab programmer that assigning zero or a negative number to an index variable at any point in a program should cause an error, I would expect them to be so puzzled by the sheer stupidity of it that they would have extreme difficulty articulating why this is a bad idea.
If unsigned indices were genuinely as good an idea as the pedants would have you believe, one might expect that in 57 years of Fortran and 30 years of Matlab, it would have been tried out and been such a great success that it would have caught on, but that is clearly not the case. Fortran, in fact, allows you to define the index range of arrays to accommodate negative indices. If this were something C-specific, one might expect that the C standard would incorporate this idea, yet it does not (§6.5.2.1 “Array subscripting” uses int) and x[-5] is perfectly valid.
The reality of doing computational linear algebra is that, although indices are non-negative integers, the arithmetic manipulation of indices in the course of executing an algorithm often, if not usually, involves not just subtraction, but negative numbers even if the eventual index computed is always non-negative. Consider, for example, the diagonal matrices formed from the stencils of finite difference methods. It would add nightmarish complexity to restrict the relative indices of the stencil to non-negative integers. What would a lower boundary check, now a trivial if(i<0) {... }, become if i were not allowed to be negative?
It might be argued that, in these cases, one “ought” to do these index computations with a signed type so as not to “pollute” the index type with negative numbers, but what exactly is the benefit of doing this? I mean, other than the smug self-satisfaction of being an über-pedant?
In practice, if you attempt to use an unsigned integer type for indices, but a signed type for index calculations, you have to add a lot of casts to silence warnings, and this quickly becomes so onerous that one either ends up either with ill-considered casts that may accidentally mask true errors, or such diligent consideration of whether a cast is truly correct that “cast contemplation” becomes a major, if not the dominant, contributor to development effort.
Again, the pedant might argue that you should have to consider these cases; that it is not a bad thing to have your development time doubled if that means it's correct. But is it “correct”? What does this “correctness” buy me? What are the actual benefits?
Well, the only one I've ever heard is this: “because size_t is the canonical type used to represent sizes in C, such as in calls to malloc(), size_t is the correct type for indexes because it guarantees that they can index any array returned by malloc() irrespective of the platform”. Really? That's it?
Leaving aside the fact that this is really an argument for choosing one of size_t and ssize_t, and I hope I've convinced you that ssize_t would be the better option, this “guarantee” is worthless, because flexibility in the representational range of a datatype used as a matrix index is, at best, useless.
The range of values that you need to use as a vector or matrix index isn't defined by what size objects you can malloc(), it's defined by the problem that you're trying to solve. So, there are actually only two scenarios that merit consideration:
Suppose that the problem of interest is such that indices need to go up to 1,000,000. You develop a solution under a 32-bit memory model where size_t is 4 bytes. Going to a 64-bit memory model where size_t is 8 bytes is essentially trivial and presents no problem (no matter the choice of datatype for indices) other than doubling the memory requirement for indices, which is not a good thing. However, going to a platform where size_t is 2 bytes is going to be very difficult: you're going to have to do quite a bit of memory management and index segmentation and manipulation to be able to solve your problem under this constraint, and direct indexing is more-or-less off the table. If you have chosen size_t for your indices, you now also have to contend with having 2 bytes automatically hacked off all of them, even in places in your code where no array is indexed; in all likelihood, this is going to make solving the issue more difficult instead of less.
So the correct answer to the question “what datatype should I use for matrix indices” is a signed integer type that is sufficiently large to represent all anticipated indices. Moreover, the correct answer to the question “what datatype should I choose for problem X” is “whatever is required by problem X” and not something provided by the compiler or platform for some other purpose.
My position is that any unsigned integral type is unsuitable for use as a general matrix or array index in computational science. My argument is essentially a utilitarian one: that pragmatism should win over naïve pedantry.
The most succinct expression of my reasoning is that, from an algorithmic perspective, it is, in general, highly desirable that indices approximate a group under addition (as closely as we can with finite size), and that using any unsigned integer type unnecessarily deprives index variables of the ability to store the additive inverse.
The argument that, formally, matrix indices are natural numbers, {1, 2, 3, ...}, holds very little weight in this argument, since C and C++ already base arrays at zero. I maintain that the ultimate reason for this is the utility of having the additive identity, 0 (zero).
Matrix-oriented languages, like Fortran and Matlab, despite basing indices at 1, do not restrict the datatype of variables used as indices strictly to the set of natural numbers. That is to say that, in these languages that are specifically and explicitly matrix- and vector-oriented, provided that an index variable is a valid index when it is used as an index, the values that it takes at other points in a program are unconstrained. I would take some convincing that the exact opposite should be the case in C or C++, yet this is precisely the position that purists ridicule me for opposing. If you were to suggest to a Fortran or Matlab programmer that assigning zero or a negative number to an index variable at any point in a program should cause an error, I would expect them to be so puzzled by the sheer stupidity of it that they would have extreme difficulty articulating why this is a bad idea.
If unsigned indices were genuinely as good an idea as the pedants would have you believe, one might expect that in 57 years of Fortran and 30 years of Matlab, it would have been tried out and been such a great success that it would have caught on, but that is clearly not the case. Fortran, in fact, allows you to define the index range of arrays to accommodate negative indices. If this were something C-specific, one might expect that the C standard would incorporate this idea, yet it does not (§6.5.2.1 “Array subscripting” uses int) and x[-5] is perfectly valid.
The reality of doing computational linear algebra is that, although indices are non-negative integers, the arithmetic manipulation of indices in the course of executing an algorithm often, if not usually, involves not just subtraction, but negative numbers even if the eventual index computed is always non-negative. Consider, for example, the diagonal matrices formed from the stencils of finite difference methods. It would add nightmarish complexity to restrict the relative indices of the stencil to non-negative integers. What would a lower boundary check, now a trivial if(i<0) {... }, become if i were not allowed to be negative?
It might be argued that, in these cases, one “ought” to do these index computations with a signed type so as not to “pollute” the index type with negative numbers, but what exactly is the benefit of doing this? I mean, other than the smug self-satisfaction of being an über-pedant?
In practice, if you attempt to use an unsigned integer type for indices, but a signed type for index calculations, you have to add a lot of casts to silence warnings, and this quickly becomes so onerous that one either ends up either with ill-considered casts that may accidentally mask true errors, or such diligent consideration of whether a cast is truly correct that “cast contemplation” becomes a major, if not the dominant, contributor to development effort.
Again, the pedant might argue that you should have to consider these cases; that it is not a bad thing to have your development time doubled if that means it's correct. But is it “correct”? What does this “correctness” buy me? What are the actual benefits?
Well, the only one I've ever heard is this: “because size_t is the canonical type used to represent sizes in C, such as in calls to malloc(), size_t is the correct type for indexes because it guarantees that they can index any array returned by malloc() irrespective of the platform”. Really? That's it?
Leaving aside the fact that this is really an argument for choosing one of size_t and ssize_t, and I hope I've convinced you that ssize_t would be the better option, this “guarantee” is worthless, because flexibility in the representational range of a datatype used as a matrix index is, at best, useless.
The range of values that you need to use as a vector or matrix index isn't defined by what size objects you can malloc(), it's defined by the problem that you're trying to solve. So, there are actually only two scenarios that merit consideration:
- size_t is big enough to represent all problem indices; and
- size_t is too small to represent all problem indices.
Suppose that the problem of interest is such that indices need to go up to 1,000,000. You develop a solution under a 32-bit memory model where size_t is 4 bytes. Going to a 64-bit memory model where size_t is 8 bytes is essentially trivial and presents no problem (no matter the choice of datatype for indices) other than doubling the memory requirement for indices, which is not a good thing. However, going to a platform where size_t is 2 bytes is going to be very difficult: you're going to have to do quite a bit of memory management and index segmentation and manipulation to be able to solve your problem under this constraint, and direct indexing is more-or-less off the table. If you have chosen size_t for your indices, you now also have to contend with having 2 bytes automatically hacked off all of them, even in places in your code where no array is indexed; in all likelihood, this is going to make solving the issue more difficult instead of less.
So the correct answer to the question “what datatype should I use for matrix indices” is a signed integer type that is sufficiently large to represent all anticipated indices. Moreover, the correct answer to the question “what datatype should I choose for problem X” is “whatever is required by problem X” and not something provided by the compiler or platform for some other purpose.
Thursday, June 5, 2014
The “Secret to Happiness” Isn't on LinkedIn
LinkedIn's machine learning algorithms have decided that I need to be emailed daily with vacuous deepities from every self-proclaimed “life coach” on their multi-million user site, when, in fact, I hold them in contempt and put them into two categories: charlatans and clowns.
The charlatans prey on the insecurities that about half of us share: we don't like our jobs, we wonder if we were “meant to do” something else, our marriages or relationships could be better, we procrastinate, we feel guilty, or trapped, or unhappy or all of the above. The “cure” to all your ills is their patent medicine, their “Secret of Happiness” and as surely as snake-oil comes in a bottle, it's always the same hackneyed aphorism: “work towards being able to “pursue your passion” as a job even if it means taking a small pay-cut”, swiftly followed by a liberal dose of loosely connected banalities.
Almost worse than the charlatans are the semi-literate deluded clowns, who — having, to all appearances, dropped out of school and never been within a bull's bellow of an introductory philosophy or psychology book — nevertheless believe they're in a position to hawk hokey cornball platitudes to the rest of us (in fairness, I don't know if this particular “coach” is a charlatan or a clown, but, either way, I wouldn't give you the gum off the back of a stamp for five hundred of her moth-eaten potboilers).
The reality is that happiness is amenable to scientific inquiry and can be studied empirically. There are legitimate “happiness researchers” in psychology, psychiatry, and neuroscience. Here's the upshot of what we know: happiness is approximately
But the really important points here are twofold:
Empirical evidence suggests that spending 15–30 minutes every day cultivating mindful awareness or practising CBT techniques will do more for your long-term happiness than any new job, spouse, or money, and certainly more than any vapid drivel peddled on LinkedIn.
The charlatans prey on the insecurities that about half of us share: we don't like our jobs, we wonder if we were “meant to do” something else, our marriages or relationships could be better, we procrastinate, we feel guilty, or trapped, or unhappy or all of the above. The “cure” to all your ills is their patent medicine, their “Secret of Happiness” and as surely as snake-oil comes in a bottle, it's always the same hackneyed aphorism: “work towards being able to “pursue your passion” as a job even if it means taking a small pay-cut”, swiftly followed by a liberal dose of loosely connected banalities.
Almost worse than the charlatans are the semi-literate deluded clowns, who — having, to all appearances, dropped out of school and never been within a bull's bellow of an introductory philosophy or psychology book — nevertheless believe they're in a position to hawk hokey cornball platitudes to the rest of us (in fairness, I don't know if this particular “coach” is a charlatan or a clown, but, either way, I wouldn't give you the gum off the back of a stamp for five hundred of her moth-eaten potboilers).
The reality is that happiness is amenable to scientific inquiry and can be studied empirically. There are legitimate “happiness researchers” in psychology, psychiatry, and neuroscience. Here's the upshot of what we know: happiness is approximately
- 50% genetic,
- 40% attitude, and
- 10% all the shit people think it's about.
But the really important points here are twofold:
- your “career”, relationships, finances, etc. basically don't matter a shit; and
- you can learn to change your “attitude”.
Empirical evidence suggests that spending 15–30 minutes every day cultivating mindful awareness or practising CBT techniques will do more for your long-term happiness than any new job, spouse, or money, and certainly more than any vapid drivel peddled on LinkedIn.
Tuesday, October 29, 2013
Seanad Reform
In the wake of the failure of the constitutional referendum to abolish the upper house, or Seanad, in Ireland, many people are clamouring for Seanad reform to make it “more democratic” and “less élitist” — in other words, they seem to want the Seanad elected by the exact same procedure that has given us an incompetent shower of party-political asshats in the Dáil for as long as anyone can remember.
What would be the point of two parallel houses directly elected by general franchise? We need only look to Washington to see how well that works. And what's wrong with élitism anyway? It's not like we need more publicans in Leinster House and fewer professors, nor does the Seanad have any real power to subvert the intentions of the democratically and directly elected Dáil in any case.
There's little disagreement that reform of both houses of the Oireachtas is highly desirable, so how about this...
The general idea here is that ministerial portfolios should be fixed prior to a general election, and three seanadóirí, suitably qualified in subject-matter related to that portfolio, would be directly elected by general franchise to a corresponding 3-member “bench”. This would mean that every minister would be “shadowed” by 3, at least somewhat knowledgeable, senators. Their job would be to directly scrutinise the his/her legislative and executive actions. Any scope or mechanism for the government to make the seanadóirí rubber-stamping cronies, or redefine their roles or the roles of ministers, is eliminated. This is the germ of an idea; we can argue the details, but what follows in an outline of how I think this could work.
First, the status quo in Ireland in relation to ministerial portfolios would be abolished with extreme prejudice. The brain-dead state of affairs where the Minister for Agriculture and Fisheries today can be replaced with a Minister for Ice Cream tommorrow — is both staggeringly wasteful and subject to populism. I originally got my ham radio license in 1988 from the “Department of Tourism, Transport, and Communications”, I now get it from the “Department of Communications, Energy & Natural Resources”, with I don't know how many “different” bizarrely-named departments, redesigned logos and letterheads, and civil service reorganizations in between.
For my idea to work, we need some kind of consistent and static — or at least not easily changed — set of ministerial portfolios, fixing ministers' roles and corresponding government departments prior to a general election. I can think of no argument against this other than that it's different from the an existing “system” that has little or nothing to recommend it.
To support this up-front fixing of ministerio-departmental portfolios, amended constitutional provisions are required: either the portfolios must be explicitly enumerated, or any change must be encumbered by requiring a supermajority in both houses, or, even better, some combination. A workable solution might be to fix the size of the cabinet at 15, and stipulate that “there shall be ministries for finance, health, education, and basket-weaving, the names and areas of responsibility of the remaining 9 ministries to be decided by two-thirds majority of both Houses, no changes to take effect until after the following general election”. That immediately does away with the stupid and wasteful gerrymandering of portfolios every year or two.
Having decided the 13 portfolios before the election, the Seanad could be constituted, with its current 60 members, as follows:
Also, in this context, a “non-Seanad political election” means a general, local, or European election anywhere in Europe. Together, these two rules remove the “wannabes” — cronies appointed to raise their political profile in preparation for running in a general election for the Dáil — and “has-beens” — cronies who just lost their seat in the Dáil (or other political assembly). The extension to Europe as a whole prevents cross-border wannabes and has-beens from the UK Parliament or Northern Ireland Assembly. Although we lack the jurisdiction to prevent a former senator running for the Northern Ireland Assembly, we can disenfranchise and disbar him/her from ever voting or being a candidate in any future election in Ireland if he/she does, or subject him/her to fines or imprisonment.
How about that for accountability? Every minister has 3 senators on his/her ass, permanently. With no way to stack the deck with has-beens and wannabes, even with no increase in power, the new Seanad would raise the standard of political discourse, while being more democratic.
The chief objection I anticipate is that voting for 13 people is too complicated for the average dimwit or that counting the votes would be too time-consuming. Fine, then: let everyone vote for just one bench, or a few benches, of their choice. That way, everyone decides what's important to them, and few people are voting purely for the sake of it on benches they have no knowledge of or don't care about. This would be at least different from the party political voting pattern that characterizes Dáil elections. That doesn't seem like a bad thing to me at all, and having a different second house is the entire point of the bicameral system.
Constitutionally and practically, the Seanad is almost entirely powerless, apart from a smattering of limited and never-used constitutional functions, such as the impeachment of a judge or the president. The Seanad can amend legislation, but the amendments are more like suggestions: they go back to the Dáil, and if the government of the day doesn't like the amendments, the bill can pass into law without Seanad support after 180 days or less, in the case of financial bills. At worst, the Seanad can delay non-financial legislation by about 9 months.
In practice, once a general election — in which members are elected to the Dáil by general franchise — is over, one of the two large parties (Fine Gael and Fianna Fáil), in conjunction with one of the smaller parties or a group of independents, will have a majority in the Dáil, which will “elect” the leaders of those parties to the cabinet, which is the new government. The leader of the largest party will be the new Taoiseach (Prime Minister), the leader of the second largest party (of the coalition making up the cabinet, not the Dáil overall) will be the Tánaiste (Deputy Prime Minister), and the cabinet positions — Ministers for Finance, Health, etc. — will be assigned to senior figures in the governing parties according to their relative strengths and the importance of the position. It is usual for the largest party to keep the Ministry for Finance for one of their own, for example.
The Taoiseach, once elected, then appoints 11 people to the Seanad. The original idea was that these would be trusted advisers and experts, but, in practice, they have always been political cronies: has-beens, who just lost their seat in the Dáil, and wannabes, who are hoping for a seat in the Dáil in the future and have been appointed to the Seanad to raise their public profile in preparation for the next general election.
The remaining 49 seanadóirí (senators) consist of 6 elected by graduates of certain Irish universities, and 43 elected from 5 so-called “vocational panels”, which consist of union, local government, and other representatives.
What would be the point of two parallel houses directly elected by general franchise? We need only look to Washington to see how well that works. And what's wrong with élitism anyway? It's not like we need more publicans in Leinster House and fewer professors, nor does the Seanad have any real power to subvert the intentions of the democratically and directly elected Dáil in any case.
There's little disagreement that reform of both houses of the Oireachtas is highly desirable, so how about this...
The general idea here is that ministerial portfolios should be fixed prior to a general election, and three seanadóirí, suitably qualified in subject-matter related to that portfolio, would be directly elected by general franchise to a corresponding 3-member “bench”. This would mean that every minister would be “shadowed” by 3, at least somewhat knowledgeable, senators. Their job would be to directly scrutinise the his/her legislative and executive actions. Any scope or mechanism for the government to make the seanadóirí rubber-stamping cronies, or redefine their roles or the roles of ministers, is eliminated. This is the germ of an idea; we can argue the details, but what follows in an outline of how I think this could work.
First, the status quo in Ireland in relation to ministerial portfolios would be abolished with extreme prejudice. The brain-dead state of affairs where the Minister for Agriculture and Fisheries today can be replaced with a Minister for Ice Cream tommorrow — is both staggeringly wasteful and subject to populism. I originally got my ham radio license in 1988 from the “Department of Tourism, Transport, and Communications”, I now get it from the “Department of Communications, Energy & Natural Resources”, with I don't know how many “different” bizarrely-named departments, redesigned logos and letterheads, and civil service reorganizations in between.
For my idea to work, we need some kind of consistent and static — or at least not easily changed — set of ministerial portfolios, fixing ministers' roles and corresponding government departments prior to a general election. I can think of no argument against this other than that it's different from the an existing “system” that has little or nothing to recommend it.
To support this up-front fixing of ministerio-departmental portfolios, amended constitutional provisions are required: either the portfolios must be explicitly enumerated, or any change must be encumbered by requiring a supermajority in both houses, or, even better, some combination. A workable solution might be to fix the size of the cabinet at 15, and stipulate that “there shall be ministries for finance, health, education, and basket-weaving, the names and areas of responsibility of the remaining 9 ministries to be decided by two-thirds majority of both Houses, no changes to take effect until after the following general election”. That immediately does away with the stupid and wasteful gerrymandering of portfolios every year or two.
Having decided the 13 portfolios before the election, the Seanad could be constituted, with its current 60 members, as follows:
- 11 to be appointed by the Taoiseach, as is currently the case;
- 10 to be elected by extending the university franchise to all university graduates, an increase of four; and
- 39 to be elected by general franchise to 13 three-member “benches” (three seats is a bench, right?), each bench directly corresponding to a ministerial portfolio.
- Candidates for each bench must be formally qualified for that bench; and
- No candidate shall be a current member of, funded by or on behalf of, nor run under the imprimatur or official endorsement of, any political party.
- No person, having been a candidate in a non-Seanad political election in the preceding 5 years, may be a senator or a candidate for election or appointment to the Seanad — the “no has-beens” rule; and
- No person, having been a senator in the last 5 years, may be a candidate in a non-Seanad political election — the “no wannabes” rule.
- No person, having lost in a Seanad election, may be appointed to the Seanad by the Taoiseach for 5 years — the “no second bite at the cherry” rule.
Also, in this context, a “non-Seanad political election” means a general, local, or European election anywhere in Europe. Together, these two rules remove the “wannabes” — cronies appointed to raise their political profile in preparation for running in a general election for the Dáil — and “has-beens” — cronies who just lost their seat in the Dáil (or other political assembly). The extension to Europe as a whole prevents cross-border wannabes and has-beens from the UK Parliament or Northern Ireland Assembly. Although we lack the jurisdiction to prevent a former senator running for the Northern Ireland Assembly, we can disenfranchise and disbar him/her from ever voting or being a candidate in any future election in Ireland if he/she does, or subject him/her to fines or imprisonment.
How about that for accountability? Every minister has 3 senators on his/her ass, permanently. With no way to stack the deck with has-beens and wannabes, even with no increase in power, the new Seanad would raise the standard of political discourse, while being more democratic.
The chief objection I anticipate is that voting for 13 people is too complicated for the average dimwit or that counting the votes would be too time-consuming. Fine, then: let everyone vote for just one bench, or a few benches, of their choice. That way, everyone decides what's important to them, and few people are voting purely for the sake of it on benches they have no knowledge of or don't care about. This would be at least different from the party political voting pattern that characterizes Dáil elections. That doesn't seem like a bad thing to me at all, and having a different second house is the entire point of the bicameral system.
Footnote on the Irish Parliamentary System
In Ireland, the Oireachtas, consists of Uachtarán na hÉireann (President of Ireland), who is directly elected to this largely ceremonial and powerless position; the 60-seat Seanad (Senate), an indirectly elected “upper house” with no real power; and the 166-seat Dáil, (House of Representatives or National Assembly), which effectively holds all legislative and executive power. This power is wielded by a 15-member cabinet (comh-aireacht, a seldom-used word since it is almost never necessary to distinguish between the cabinet and the Government, or Rialtas), elected by the Dáil from amongst their membership, with a seldom-exercised constitutional provision to have up to two seanadóirí (senators).Constitutionally and practically, the Seanad is almost entirely powerless, apart from a smattering of limited and never-used constitutional functions, such as the impeachment of a judge or the president. The Seanad can amend legislation, but the amendments are more like suggestions: they go back to the Dáil, and if the government of the day doesn't like the amendments, the bill can pass into law without Seanad support after 180 days or less, in the case of financial bills. At worst, the Seanad can delay non-financial legislation by about 9 months.
In practice, once a general election — in which members are elected to the Dáil by general franchise — is over, one of the two large parties (Fine Gael and Fianna Fáil), in conjunction with one of the smaller parties or a group of independents, will have a majority in the Dáil, which will “elect” the leaders of those parties to the cabinet, which is the new government. The leader of the largest party will be the new Taoiseach (Prime Minister), the leader of the second largest party (of the coalition making up the cabinet, not the Dáil overall) will be the Tánaiste (Deputy Prime Minister), and the cabinet positions — Ministers for Finance, Health, etc. — will be assigned to senior figures in the governing parties according to their relative strengths and the importance of the position. It is usual for the largest party to keep the Ministry for Finance for one of their own, for example.
The Taoiseach, once elected, then appoints 11 people to the Seanad. The original idea was that these would be trusted advisers and experts, but, in practice, they have always been political cronies: has-beens, who just lost their seat in the Dáil, and wannabes, who are hoping for a seat in the Dáil in the future and have been appointed to the Seanad to raise their public profile in preparation for the next general election.
The remaining 49 seanadóirí (senators) consist of 6 elected by graduates of certain Irish universities, and 43 elected from 5 so-called “vocational panels”, which consist of union, local government, and other representatives.
Tuesday, August 27, 2013
Installer Tips for Open Source Developers
So, I've recently had to install a lot of software from source. I used to do this a lot — back in the days when I was a Linux hobbyist and had nothing better to do — but in recent years, I have tended to just live with whatever versions of software libraries come with whatever Linux distro I happen to be using on a particular machine, usually the latest Ubuntu LTS (although I've used Slackware, Red Hat, SuSE, Mandriva, Yellow Dog, and CentOS in addition to Irix, Solaris, and FreeBSD).
One thing that hasn't changed one iota in (almost) twenty years is the utter cluelessness of niche developers when it comes to packaging their software, and it's driving me nuts. Just because your software isn't totally mainstream doesn't mean you can subject your (potential) users to an installation nightmare of manually editing Makefiles and making decisions about minutiae of configuration and installation.
The rules are very, very, simple indeed. Everyone who's ever installed anything from source knows the rules, so why the hell don't niche developers?
Here it is. The standard way of installing from source is
After I download a (well-behaved tarball), I expect to be able to do this:
$ tar xzf foo-1.2.3.tar.gz
$ cd foo-1.2.3
$ ./configure
$ make
$ make test|check
$ sudo make install
There are a small number of variations, of course, you can use 7zip or bzip2 instead of gzip, and so on. At a push, you can inflict CMake or SCons on me, but, frankly, I don't care how much of a pain in the hole the Autotools are (and, as a developer, I hate them), ideally you should use them because that's what your users expect, and they don't really care if it takes you a day of frustration to figure out. If that's significant in the context of your overall software development, it's probably not worth installing your software in the first place.
If you insist on inflicting your egotistical notions of how installation from source should work, the one thing you absolutely cannot do is ignore now well-established conventions about where stuff goes: absent any explicit instruction from me, you install your shit in the appropriate directories in /usr/local and nowhere else. I don't care that it's easier for you to just copy great gobs of shit into /opt/IAmSoImportant and leave all the painful configuration to me, and for the love of all that is good and right, if you install anything directly in /usr, interfering with the packaging system of my distro, I will hunt you down like a rabid dog and hammer a mechanical pencil into your eye with your detached and still-bleeding leg. I know it's hard to decide whether you should put your headers directly in /usr/local/include or in a subdirectory thereof, or your libraries in /usr/local/lib or in a subdirectory thereof, and it's often subtly debatable how “platform independent” that file really is, so whether it should go in /usr/local/share/foo or somewhere else is, in the end, a judgement call. I know. I sympathise. I really do. But if you can't decide, and you wrote the damn thing, how the hell do you expect me to?
In short, if you want your software to ever have the slightest chance of “catching on” and becoming popular, you can't just do the fun stuff, you have to suck it up, make a decision or two, and do the mundane and tedious stuff that makes installation easy for your users and packaging easy for distributions.
End of rant :o)
One thing that hasn't changed one iota in (almost) twenty years is the utter cluelessness of niche developers when it comes to packaging their software, and it's driving me nuts. Just because your software isn't totally mainstream doesn't mean you can subject your (potential) users to an installation nightmare of manually editing Makefiles and making decisions about minutiae of configuration and installation.
The rules are very, very, simple indeed. Everyone who's ever installed anything from source knows the rules, so why the hell don't niche developers?
Here it is. The standard way of installing from source is
After I download a (well-behaved tarball), I expect to be able to do this:
$ tar xzf foo-1.2.3.tar.gz
$ cd foo-1.2.3
$ ./configure
$ make
$ make test|check
$ sudo make install
There are a small number of variations, of course, you can use 7zip or bzip2 instead of gzip, and so on. At a push, you can inflict CMake or SCons on me, but, frankly, I don't care how much of a pain in the hole the Autotools are (and, as a developer, I hate them), ideally you should use them because that's what your users expect, and they don't really care if it takes you a day of frustration to figure out. If that's significant in the context of your overall software development, it's probably not worth installing your software in the first place.
If you insist on inflicting your egotistical notions of how installation from source should work, the one thing you absolutely cannot do is ignore now well-established conventions about where stuff goes: absent any explicit instruction from me, you install your shit in the appropriate directories in /usr/local and nowhere else. I don't care that it's easier for you to just copy great gobs of shit into /opt/IAmSoImportant and leave all the painful configuration to me, and for the love of all that is good and right, if you install anything directly in /usr, interfering with the packaging system of my distro, I will hunt you down like a rabid dog and hammer a mechanical pencil into your eye with your detached and still-bleeding leg. I know it's hard to decide whether you should put your headers directly in /usr/local/include or in a subdirectory thereof, or your libraries in /usr/local/lib or in a subdirectory thereof, and it's often subtly debatable how “platform independent” that file really is, so whether it should go in /usr/local/share/foo or somewhere else is, in the end, a judgement call. I know. I sympathise. I really do. But if you can't decide, and you wrote the damn thing, how the hell do you expect me to?
In short, if you want your software to ever have the slightest chance of “catching on” and becoming popular, you can't just do the fun stuff, you have to suck it up, make a decision or two, and do the mundane and tedious stuff that makes installation easy for your users and packaging easy for distributions.
End of rant :o)
Thursday, August 8, 2013
Rapid Charging Electric Cars
One of the things that comes up repeatedly with electric cars is how fast you can charge them. The current answer is “overnight” or, at least, on the order of hours. Some more bullish electric car proponents argue that charging stations could be built that would reasonably match gasoline filling speeds. I don't think that's plausible.
The EPA limits gasoline filling speeds to 10 gpm (gallons per minute), or 0.63 l/s (liters per second). The volumetric energy density of gasoline is about 36 MJ/l (megajoules per liter), which means that, at the gas station, energy is flowing into your tank at a rate of 36 MJ/l * 0.63 l/s = 22.7 MJ/s = 22.7 MW (megawatts).
Now, suppose that the battery-to-wheels efficiency of an electric car is five times the tank-to-wheels efficiency of a gasoline car, which is a fairly reasonable assumption. Then we only need a charging power of 22.7/5 = 4.5 MW. Generously supposing 90% efficiency from grid to battery, we need “only” draw five million watts off the power grid.
That is an absolutely vast amount of electric power: not much less than is needed to supply 4,000 American homes. This is not a plug-in device.
A reasonable rule of thumb for distribution level electricity supply is “100VA/kV” in other words, if you were to be a 10kV primary customer, you can expect to be able to draw at most 1 MVA (the difference between VA and watts is not important for the current discussion). On that basis, the 5MW charging station will need to be a 69 kV subtransmission customer of the local power utility. One charging station, not a gas station forecourt with 8 of them.
Now, let's talk about slew rate. You can't just turn on a 5 MW load (for want of a round number) like a 60W lightbulb. Call your local power company and ask them how quickly they would allow a subtransmission customer to turn on a 5 MW load. The answer cannot be faster than they can spin up a gas turbine. The slew rate of a “hot” GE gas turbine generator is, at most, 5%-per-minute. In other words, to be able to even ramp-up a 5MW charger to full power in one minute would take 100MW of spinning reserve, and only after that have you hit gas pump-equivalent power.
Let's look at that another way. Take the 85kWh (kilowatt hour) battery in the top-end Tesla Model S. 85kWh is 306MJ. Suppose you want to charge that sucker in a minute flat, which is not unreasonable given its range of 265 miles: one minute would give you ten gallons of gas, at least enough to run a modern luxury car for that distance. To supply 306 MJ in 60s is 306/60 = 5.1 MW. Pretty much the same number.
In synopsis, refuelling an electric car at the same rate as a gas pump, any way you look at it, requires something of the order of 5 MW of electric power. This is, practically speaking, impossible. Notice that I haven't mentioned cost. Economically speaking, it is utterly beyond any reason: the charger alone would cost millions. Recharging an electric car at even ten percent of gas-pump equivalent speeds (requiring “only” a half-megawatt charger), presents enormous technical challenges in electricity supply.
In short, the technical challenges of recharging an electric car on consumer-acceptable timescales is almost nothing to do with the car or its battery.
The EPA limits gasoline filling speeds to 10 gpm (gallons per minute), or 0.63 l/s (liters per second). The volumetric energy density of gasoline is about 36 MJ/l (megajoules per liter), which means that, at the gas station, energy is flowing into your tank at a rate of 36 MJ/l * 0.63 l/s = 22.7 MJ/s = 22.7 MW (megawatts).
Now, suppose that the battery-to-wheels efficiency of an electric car is five times the tank-to-wheels efficiency of a gasoline car, which is a fairly reasonable assumption. Then we only need a charging power of 22.7/5 = 4.5 MW. Generously supposing 90% efficiency from grid to battery, we need “only” draw five million watts off the power grid.
That is an absolutely vast amount of electric power: not much less than is needed to supply 4,000 American homes. This is not a plug-in device.
A reasonable rule of thumb for distribution level electricity supply is “100VA/kV” in other words, if you were to be a 10kV primary customer, you can expect to be able to draw at most 1 MVA (the difference between VA and watts is not important for the current discussion). On that basis, the 5MW charging station will need to be a 69 kV subtransmission customer of the local power utility. One charging station, not a gas station forecourt with 8 of them.
Now, let's talk about slew rate. You can't just turn on a 5 MW load (for want of a round number) like a 60W lightbulb. Call your local power company and ask them how quickly they would allow a subtransmission customer to turn on a 5 MW load. The answer cannot be faster than they can spin up a gas turbine. The slew rate of a “hot” GE gas turbine generator is, at most, 5%-per-minute. In other words, to be able to even ramp-up a 5MW charger to full power in one minute would take 100MW of spinning reserve, and only after that have you hit gas pump-equivalent power.
Let's look at that another way. Take the 85kWh (kilowatt hour) battery in the top-end Tesla Model S. 85kWh is 306MJ. Suppose you want to charge that sucker in a minute flat, which is not unreasonable given its range of 265 miles: one minute would give you ten gallons of gas, at least enough to run a modern luxury car for that distance. To supply 306 MJ in 60s is 306/60 = 5.1 MW. Pretty much the same number.
In synopsis, refuelling an electric car at the same rate as a gas pump, any way you look at it, requires something of the order of 5 MW of electric power. This is, practically speaking, impossible. Notice that I haven't mentioned cost. Economically speaking, it is utterly beyond any reason: the charger alone would cost millions. Recharging an electric car at even ten percent of gas-pump equivalent speeds (requiring “only” a half-megawatt charger), presents enormous technical challenges in electricity supply.
In short, the technical challenges of recharging an electric car on consumer-acceptable timescales is almost nothing to do with the car or its battery.
Wednesday, July 17, 2013
NatGeo on Biofuels
What breakthroughs do biofuels need? asks National Geographic.
The real breakthrough would be recognition that biofuels simply cannot reasonably be expected to address a significant fraction of our energy needs. The only breakthrough worth having would be a tenfold increase in the solar efficiency of photosynthesis, which would only be possible through advanced genetic engineering, is very far beyond our current capabilities, and would be vehemently opposed by environmentalists.
The basic problem is that photosynthesis is horribly inefficient in converting sunlight to usable fuel. The maximum theoretical energy conversion efficiency of sunlight to biomass, not useful fuel, is just 6% [Zhu et al., 2008]; real efficiencies are considerably lower.
Brazil's sugarcane ethanol production is the absolute gold standard for large-scale biofuel production. The Brazilians started this bandwagon in the 70's and have aggressively optimized their ethanol production for 40 years. The most bullish prediction is that — if they keep improving at the same rate as they have in the past — they'll be able to average 9,000 liters of ethanol per hectare per year by 2018 [Goldemberg, 2008].
What's the energy content of 9,000 liters of ethanol? The highest value I could find is 23.4 MJ/l (megajoules per liter), corresponding to the HHV (higher heating value) of anhydrous ethanol. This means that 9 kl (kiloliters) of ethanol yields at most 211 GJ (gigajoules) of thermal energy on combustion.
Now, how much sunlight falls on a hectare? In energy slang a “sun” is 1 kW/m^2 (kilowatt per square meter). That's about the peak insolation (energy density on the ground from the sun) at noon at the equator, but of course, the sun doesn't shine at night, insolation falls with latitude, and there's seasonal variation. An insolation map of Brazil suggests that a reasonable value for the sum of the insolation over a year may be up to 2,000 kWh/m^2 (kilowatt hours per square meter), so let's be super-generous to biofuels and use a stingy figure of 1,000 kWh/m^2, which is more like the correct value for Ireland than Brazil, and equates to 36,000 GJ/ha.
So, every hectare gets at least 36,000 GJ of sunlight and produces at most 211 GJ of ethanol. That's an energy conversion efficiency of, at best, 211/36000 or less than 0.6%. A fair figure (using the LHV of ethanol, production of 7kl/ha, and 1,750 kWh/m^2) would be less than half of that. I think it's fair to say that the solar-to-liquid-fuel energy conversion efficiency of sugarcane ethanol production is currently no more than one quarter of one percent, less by the time the input energy necessary to grow, harvest, ferment, and distill the ethanol (at least one tenth of the energy produced) is accounted for.
You can do far better than this generating hydrogen in your backyard using a modern solar PV panel (20% efficient) to power a commercially available electrolyser (73% efficient). Under reasonable assumptions, today you can produce hydrogen via solar panels and electrolysis with more than 50 times the efficiency of sugarcane ethanol production. I'm not arguing for hydrogen-powered cars, merely illustrating the horrible inefficiency of ethanol production.
Speaking of cars, to ram the point home, let's suppose that we could magically transform all of the vehicles in the United States into flex-fuel vehicles capable of running on 100% ethanol. Let's further suppose that we could out-do the future Brazilians at their own game and obtain fantastic yields of 10 kl/ha (or 1 l/m^2). Let's further suppose that we could substitute ethanol 1:1 for gasoline, meaning that we would need 500 billion liters of ethanol per year just for gasoline, never mind other energy needs. How much land would that require? About 50 million hectares. That's about half the total area of the United States or about three times its arable land area. If you use actual figures for US corn ethanol production (about 3,750 l/ha)? You need 8 times the arable land area of the USA.
So when I see headlines like this, I say, “So what, it's 1% efficient?”
I don't know how we can address our need for a gasoline substitute (easily transportable, high energy density, short “recharge” time, etc.), but it seems pretty clear that biofuels are merely a distraction.
The real breakthrough would be recognition that biofuels simply cannot reasonably be expected to address a significant fraction of our energy needs. The only breakthrough worth having would be a tenfold increase in the solar efficiency of photosynthesis, which would only be possible through advanced genetic engineering, is very far beyond our current capabilities, and would be vehemently opposed by environmentalists.
The basic problem is that photosynthesis is horribly inefficient in converting sunlight to usable fuel. The maximum theoretical energy conversion efficiency of sunlight to biomass, not useful fuel, is just 6% [Zhu et al., 2008]; real efficiencies are considerably lower.
Brazil's sugarcane ethanol production is the absolute gold standard for large-scale biofuel production. The Brazilians started this bandwagon in the 70's and have aggressively optimized their ethanol production for 40 years. The most bullish prediction is that — if they keep improving at the same rate as they have in the past — they'll be able to average 9,000 liters of ethanol per hectare per year by 2018 [Goldemberg, 2008].
 |
| Brazilian Ethanol Yield Over Time |
Now, how much sunlight falls on a hectare? In energy slang a “sun” is 1 kW/m^2 (kilowatt per square meter). That's about the peak insolation (energy density on the ground from the sun) at noon at the equator, but of course, the sun doesn't shine at night, insolation falls with latitude, and there's seasonal variation. An insolation map of Brazil suggests that a reasonable value for the sum of the insolation over a year may be up to 2,000 kWh/m^2 (kilowatt hours per square meter), so let's be super-generous to biofuels and use a stingy figure of 1,000 kWh/m^2, which is more like the correct value for Ireland than Brazil, and equates to 36,000 GJ/ha.
 |
| Yearly Sum of Global Irradiance |
So, every hectare gets at least 36,000 GJ of sunlight and produces at most 211 GJ of ethanol. That's an energy conversion efficiency of, at best, 211/36000 or less than 0.6%. A fair figure (using the LHV of ethanol, production of 7kl/ha, and 1,750 kWh/m^2) would be less than half of that. I think it's fair to say that the solar-to-liquid-fuel energy conversion efficiency of sugarcane ethanol production is currently no more than one quarter of one percent, less by the time the input energy necessary to grow, harvest, ferment, and distill the ethanol (at least one tenth of the energy produced) is accounted for.
You can do far better than this generating hydrogen in your backyard using a modern solar PV panel (20% efficient) to power a commercially available electrolyser (73% efficient). Under reasonable assumptions, today you can produce hydrogen via solar panels and electrolysis with more than 50 times the efficiency of sugarcane ethanol production. I'm not arguing for hydrogen-powered cars, merely illustrating the horrible inefficiency of ethanol production.
Speaking of cars, to ram the point home, let's suppose that we could magically transform all of the vehicles in the United States into flex-fuel vehicles capable of running on 100% ethanol. Let's further suppose that we could out-do the future Brazilians at their own game and obtain fantastic yields of 10 kl/ha (or 1 l/m^2). Let's further suppose that we could substitute ethanol 1:1 for gasoline, meaning that we would need 500 billion liters of ethanol per year just for gasoline, never mind other energy needs. How much land would that require? About 50 million hectares. That's about half the total area of the United States or about three times its arable land area. If you use actual figures for US corn ethanol production (about 3,750 l/ha)? You need 8 times the arable land area of the USA.
So when I see headlines like this, I say, “So what, it's 1% efficient?”
I don't know how we can address our need for a gasoline substitute (easily transportable, high energy density, short “recharge” time, etc.), but it seems pretty clear that biofuels are merely a distraction.
Sunday, June 30, 2013
Holy Fukushima: Scaremongering is Everywhere!
It seems like this kind of thing has been doing the rounds:
I got a message on Facebook saying “I'd really like you to do a blog piece on this”, so here it is.
The above image is, in fact, an ocean wave amplitude graphic for the April 2011 Fukushima earthquake from NOAA. It has nothing to do with the “fallout” from the Fukushima Daiichi nuclear plant. I repeat: it has nothing whatever to do with nuclear radiation of any kind — it is an ocean wave amplitude graphic. Somebody took this innocent graphic and maliciously emblazoned it with a scaremongering lie. A slew of ignorant anti-nuclear Internet Luddites then reposted this complete fabrication, which has been swallowed wholesale by some of the more gullible members of the public.
In many cases, the above graphic has been replaced by this one:
Now, this one is actually a particle simulation from the New Zealand based ASR Ltd., a marine consulting company. If you actually go to the original page, it says (in their block capitals): “THIS IS NOT A REPRESENTATION OF THE RADIOACTIVE PLUME CONCENTRATION”. What this graphic actually tells us — if their computer simulation is accurate and reliable — is that, in the year after Fukushima, nothing (radioactive or otherwise) in the ocean surface currents could possibly have gotten much further than about halfway across the Pacific Ocean, which is quite a different thing from what the scaremongers would have you believe, and says nothing whatsoever about dilution or concentration. This was a publicity stunt by a private company showcasing their technology; it is neither peer-reviewed science, nor the report of a competent panel of experts.
So, is radiation from Fukushima killing Americans?
If you look at similar peer-reviewed science [Behrens et al., 2012], whose graphics have also been used for scaremongering, what you find is that the radioactivity of 137Cs — everyone's favorite radioisotope — off the coast of California (blue box IV) due to Fukushima peaks at about 1.2 Bq/m3, while further North it might peak around 2 (cyan box II). To put this in perspective, the background of 137Cs in the Pacific is about 3 Bq/m3, and the background level of radon in the air averages 5–15 Bq/m3, depending on where you live. In other words, this kind of increase — another couple of becquerels per cubic meter — isn't going to make a whole lot of difference to anyone.
The expert consensus is similarly undramatic: by far the most pessimistic part of the WHO's assessment is that the lifetime risk of thyroid cancer for a 1 year old female in the most affected parts of Fukushima prefecture may increase by 70%. Wow! 70%. But here's the thing: the baseline lifetime risk of thyroid cancer for women is 0.75%, 70% of that, the additional lifetime risk, is just 0.5%. I'm not saying I'd like my risk of some kind of cancer to increase by 0.5%, but it's not anything that I'm going to get my knickers in a knot over either.
If you look around, you'll find out that the total Fukushima release was 900 PBq — by any standard an enormous amount of radiation — about one sixth of a Chernobyl, about equivalent to the fallout from a 2Mt nuclear warhead, or approximately bugger all compared to what the Americans, the British, the French, and the Russians were doing throughout the 50's, 60's and 70's in the Pacific, Siberia, and the Nevada desert.
You'll also find that a reasonable estimate of the total number of additional cancer-related deaths attributable to Fukushima is about 130. That's less than half the number of coal-mining deaths over the last 10 years in the US, less than a day-and-a-half's worth of road traffic fatalities, or a few weeks of coal-mining deaths in China. In other words, also bugger all.
The reality is that nuclear power plants are actually pretty safe in the grand scheme of things, it's just that when there is an accident, it's big and it makes a big splash on the news. It's a bit like the way a plane crash that kills 300 people is a major news event, but the 300 people who die on our roads every few days in an unnoticeable trickle never get on CNN.
So, to answer the question, the danger to Americans from Fukushima is essentially zero. If you're going to start washing your vegetables in filtered water — as some of the sensationalist anti-nuclear liars in the lede would have you do — think again… with a little more skepticism and balance.
 |
| Bogus Fukushima Radiation Map #1 |
The above image is, in fact, an ocean wave amplitude graphic for the April 2011 Fukushima earthquake from NOAA. It has nothing to do with the “fallout” from the Fukushima Daiichi nuclear plant. I repeat: it has nothing whatever to do with nuclear radiation of any kind — it is an ocean wave amplitude graphic. Somebody took this innocent graphic and maliciously emblazoned it with a scaremongering lie. A slew of ignorant anti-nuclear Internet Luddites then reposted this complete fabrication, which has been swallowed wholesale by some of the more gullible members of the public.
In many cases, the above graphic has been replaced by this one:
 |
| Bogus Fukushima Radiation Map #2 |
Now, this one is actually a particle simulation from the New Zealand based ASR Ltd., a marine consulting company. If you actually go to the original page, it says (in their block capitals): “THIS IS NOT A REPRESENTATION OF THE RADIOACTIVE PLUME CONCENTRATION”. What this graphic actually tells us — if their computer simulation is accurate and reliable — is that, in the year after Fukushima, nothing (radioactive or otherwise) in the ocean surface currents could possibly have gotten much further than about halfway across the Pacific Ocean, which is quite a different thing from what the scaremongers would have you believe, and says nothing whatsoever about dilution or concentration. This was a publicity stunt by a private company showcasing their technology; it is neither peer-reviewed science, nor the report of a competent panel of experts.
So, is radiation from Fukushima killing Americans?
If you look at similar peer-reviewed science [Behrens et al., 2012], whose graphics have also been used for scaremongering, what you find is that the radioactivity of 137Cs — everyone's favorite radioisotope — off the coast of California (blue box IV) due to Fukushima peaks at about 1.2 Bq/m3, while further North it might peak around 2 (cyan box II). To put this in perspective, the background of 137Cs in the Pacific is about 3 Bq/m3, and the background level of radon in the air averages 5–15 Bq/m3, depending on where you live. In other words, this kind of increase — another couple of becquerels per cubic meter — isn't going to make a whole lot of difference to anyone.
The expert consensus is similarly undramatic: by far the most pessimistic part of the WHO's assessment is that the lifetime risk of thyroid cancer for a 1 year old female in the most affected parts of Fukushima prefecture may increase by 70%. Wow! 70%. But here's the thing: the baseline lifetime risk of thyroid cancer for women is 0.75%, 70% of that, the additional lifetime risk, is just 0.5%. I'm not saying I'd like my risk of some kind of cancer to increase by 0.5%, but it's not anything that I'm going to get my knickers in a knot over either.
If you look around, you'll find out that the total Fukushima release was 900 PBq — by any standard an enormous amount of radiation — about one sixth of a Chernobyl, about equivalent to the fallout from a 2Mt nuclear warhead, or approximately bugger all compared to what the Americans, the British, the French, and the Russians were doing throughout the 50's, 60's and 70's in the Pacific, Siberia, and the Nevada desert.
You'll also find that a reasonable estimate of the total number of additional cancer-related deaths attributable to Fukushima is about 130. That's less than half the number of coal-mining deaths over the last 10 years in the US, less than a day-and-a-half's worth of road traffic fatalities, or a few weeks of coal-mining deaths in China. In other words, also bugger all.
The reality is that nuclear power plants are actually pretty safe in the grand scheme of things, it's just that when there is an accident, it's big and it makes a big splash on the news. It's a bit like the way a plane crash that kills 300 people is a major news event, but the 300 people who die on our roads every few days in an unnoticeable trickle never get on CNN.
So, to answer the question, the danger to Americans from Fukushima is essentially zero. If you're going to start washing your vegetables in filtered water — as some of the sensationalist anti-nuclear liars in the lede would have you do — think again… with a little more skepticism and balance.
Subscribe to:
Posts (Atom)