rlucas.net: The Next Generation Rotating Header Image

ancient

FIX: Can't locate object method "select_val" via package "DBI::st" under Class::DBI 0.95

[warning: see updates below]

Between Class::DBI 0.94 and 0.95, something changed causing my classes that override db_Main() to set the db connection (instead of, e.g. using set_db) to stop working.  Specifically, when their sequence functions were invoked, I got an error of:

 

Can't locate object method “select_val” via package “DBI::st” (perhaps you forgot to load “DBI::st”?) at /usr/lib/perl5/site_perl/5.6.1/Class/DBI.pm line …

 

I was able to replicate this on Mac OS X 10.2 with Perl 5.6.0 and Red Hat 9 with 5.6.1 (don't ask about the latter version magic…).

 

If you get select_val errors with Class::DBI 0.95, here are two workarounds:

 

 

I am not sure why this is (comments are welcome) and have submitted a bug to the developers as CPAN #5522.

Update: Thanks to Tony Bowden, maintainer of Class::DBI, for his reply:

 

Full context and any attached attachments can be found at:
<URL: http://rt.cpan.org/NoAuth/Bug.html?id=5522 >

On Mon, Mar 01, 2004 at 06:42:38PM -0500, Guest via RT wrote:
> In Class::DBI 0.95, setting up the DB connection by overriding db_Main
> breaks Ima::DBI->select_val and the methods that rely on it (like sequence
> and count_all)

You need to call Ima::DBI->connect rather than DBI->connect in your
overriden db_Main.

Tony

 

Still not certain, though, why it is that it breaks in 0.95 and not 0.94.

Update: Thanks again to Tony, who writes:

The reason this broke in going from 0.94 to 0.95, btw, is that the
select_val stuff was only added to Ima::DBI recently, so Class::DBI .94
didn't use these, but instead rolled its own long winded versions.

0.95 uses the new methods in Ima::DBI and is a lot better off for it! 🙂

Tony

Update 23 April 2004: Things are all wacky now.  You should be careful and should probably NOT do any of the above.  See the 06 April 2004 exchanges on the CDBI mailing list.  If you do what is described above you stand to get reconnections that will mess up things like auto_increments in MySQL.  At present the issue of how to deal with db_Main() seems unresolved.

Once Upon A Time

[Update: As is often the case when lots and lots of people (say, the whole Internet)
look at a problem, I came to this conclusion independently along with a
whole bunch of other folks.  I wrote this freshman effort at
blogging prior to becoming aware of the “Eternal September” concept;
however, this trope of pre/post-1993 Internet quality is much more
concisely described by the “Eternal September” entry in Wikipedia:
http://en.wikipedia.org/wiki/Eternal_September .  My take on it
doesn't put as much blame directly on AOL users as the folk wisdom of
Eternal September does; I try to look at structural differences in the
modes of communication and speculate as to their effects on the types
of interactions that went on.]

Once upon a time, the Internet was cool (circa pre-1993). At that time
there was a lot of info with a decent signal to noise ratio, and a lot
of knowledgable people, You could read the FAQs for a newsgroup on a
subject (anything from hang gliding to Germany) and get a fairly good
dose of knowledge on the topic, as well as a direct line to a bunch of
people who knew it well. Is there a way to get something as cool as
that back out of today's incarnation of the Internet (that is, the
largely Web-mediated experience)? I hold that maybe there is some hope
and that we can get the Internet back to being somewhat collaborative
and useful again.

If the Internet was so grand, what did people
do with it back then? There was the normal Internet stuff that still
goes on today and will probably go on forever: email and FTP, which
respectively served the most personal and most technical needs of its
users (sending letters and distributing software). There was real-time
chatting of various types, much as there is today. But the big
difference in the way people interacted then and now is the difference
between Usenet and the Web.

Usenet (a.k.a. netnews or
newsgroups) provided for the syndication of so-called “news” messages
grouped into subject-matter categories. In practice, these newsgroups
weren't really news per se. They were rather forums for discussion and
debate by people, often quite knowledgable people, about defined
subject areas (of all sorts, but most commonly political/religious
debate, hobbies, and computer/technical issues). People built up their
reputations by contributing constructively to these discussions but the
most presitigious thing you could do within the context of a newsgroup
was to help maintain its FAQ. The Frequently Asked Questions list was
kind of a “greatest hits” of the newsgroup's content. Most of the
active newsgroups had these FAQs, and they were routinely made
available in the context of the newsgroup itself as well as being
archived and distributed as ends in themselves. The maintainers of an
FAQ of course had to be able contributors who would structure and even
add novel material to the FAQ, but the document really represented a
collaborative effort of the group's active members, and was often
largely paraphrased or excerpted from newsgroup postings (with
attribution; another honor for the constructive group member).

(There
was of course no such thing as a newsgroup that had only one member who
wrote the FAQ based upon his own discussion with himself and the
questions he had answered. The idea would be preposterous; newsgroups
were collaborative centers.)

(Note that the kind of knowledge
I'm discussing here is not the easy kind, like stock quotes, movie
times, sports scores, etc., which various companies have already
handled quite well [and which, I may add, were not nearly so easily
available during the Usenet era]. I call that the “easy” kind of
information because it's easy to imagine the SQL statement that
retrieves it, e.g. select showtime, location from movie_showings where
film_id = 38372 and city_name = 'boston'. I'm more interested in domain
knowledge of a particular field, such as “what are some good books I
should read to learn about hang gliding,” or “what does it mean if
program foo version 4.21 says 'error xyz-2?'”)

Sometime after
1993 a bunch of things started happening: commercial spam began to fill
up Usenet and folks' email boxes; waves of the uninitiated began
incurring the wrath of old-timers by their breaches of netiquette,
leading to a general lowering of the signal-to-noise ratio; and, of
course, people got turned on to this whole idea of the Web. Here was a
medium in which anyone could become a publisher! If you were expert on
a topic, or if you had a cool digital photo, or if you just happened to
know HTML, you could publish a Web site and become sort of famous! Of
course, this was a pain in the ass: posting on Usenet just meant typing
an email message, but having a web page required knowing and doing a
lot of tedious but not very interesting stuff, so you really had to
have some time to put into it.

However, the Web had pictures and
clicking with the mouse, while Usenet had boring words and typing —
and AOL users were starting to come onto the Internet. So the Web took
over.

The dominant mode for interaction on the Internet — but
more importantly, for publishing of subject-matter knowledge — moved
away from Usenet to the Web. (Of course, Usenet is still around, and
the newsgroups generally put their FAQs on the Web, but a newcomer to
the Internet might never even hear of Usenet during his Web-mediated
experience.) Rather than posting an article to a group and waiting to
read other articles posted in response, you now published a “site” and
counted how many visitors came. (Plus, you could enjoy hours on the web
without ever using your keyboard, which meant of course that its users
were even physically disconnected from the means of actually inputting
any information.)

Everyone who was an aspirant to Web fame and
had an interest in model trains, say, would create his own model trains
Web site, provide his own set of (supposedly) interesting content, and,
often, maintain his own FAQ of questions asked of him by visitors to
the site. At first, these aspirants were individuals, but soon enough
affinity groups or associations and commerical interests got involved,
doing basically the same thing. Perhaps you see where I am going with
this, gentle reader. The way in which personal knowledge was packaged
up and distributed became centered on the individual, and the
relationship changed from one of collaboration between peers to one of
publisher and reader.

A well-known lament about web publishing
is that unlike print publishing, the cost is so low as to admit
amateurs, crazies, and just plain bad authors — anyone with sufficient
motivation to brave the arcana of FTP and HTML. On the other hand, I
have just complained that the model simultaneously changed from a
peer-to-peer to a client-server relationship. Could it be that both of
these charges are true? It seems this would be the worst of both
worlds: not only are people no longer as engaged in the constructive
addition to the commons, but those that control the production and
distribution of knowledge aren't even filtered out by the requirements
of capital investment. It's like creating a legislature by taking the
worst parts each from the House and Senate. Sadly, this describes much
of the past ten years of the Internet's history.

However, there
is some hope. Whereas previously, “anyone” could have a Web site but
precious few put in the many hours it required in practice, the promise
of Weblogs is to actually open Web publishing to “anyone.” This won't
filter out the crazies, but at least it won't artificially inflate
their importance by raising the bar just high enough to keep everyone
else out. Comment forums, content-management systems, Wikis,
trackbacks, and the like are helping to re-enable the sort of
collaboration that made the Usenet system work.

Bottom line: it rather feels like we're almost back to 1993.

Next time: future directions, pitfalls, and why blogging (alone) is not the answer.

FIX: Compiling SWI-Prolog on Mac OS X 10.2

SWI-Prolog is available as prepackaged binaries for Mac OS X 10.3+, but
not for 10.2.  If you try and install the 10.3 binary package, you
will get errors (at least, I did).  The answer is to compile from
source.  You are probably compile-savvy if you are looking to
install a Prolog interpreter, but if not, it's a fairly painless
./configure, make, make install process.

1. However, the docs warn that you'll want readline and a number of
other libraries installed.  There are some binary packages on the
SWI-Prolog site.  If you want to use those, and you don't have any
other versions of the libraries, so be it — but I would recommend
using Fink instead, so that you can install the most up to date
versions.

2. Especially if using Fink, be sure to alert the ./configure script to
the locations by including LDFLAGS=”-I /sw/include” and CFLAGS=”-L
/sw/lib” (or wherever).

For me, all it took was pointing the configure script to the /sw tree and it compiled with no further questions.

FIX: GIMP can't open PostScript (PS, EPS, PDF) files under Windows

The GIMP (GNU Image Manipulation Program) is a neat tool for people with needs too casual or cheap for PhotoShop, but too much for various paintbrush type tools.

However, if you install the GIMP under Windows 2000, like me, EPS or PS PostScript files will not open properly, instead barfing with:

Error opening file: C:tempmyfile.eps

PS: Can't interprete file [sic]

You'll need to do the following to make it work:

1. Install GhostScript for Windows.

http://www.cs.wisc.edu/~ghost/doc/AFPL/get811.htm

2. Install the GIMP.

http://www2.arnes.si/~sopjsimo/gimp/

3. Set your environment variables to include:

GS_PROG=c:pathtoandfilename.exe

GS_LIB=c:justthepathlib

Typical paths in which to find your GS stuff after a default install might be C:gsgs8.11bingswin32c.exe and C:gsgs8.11lib

(One way to get your environment set in Windows is Start: Settings:Control Panel:System:Advanced:Environment Variables.  In non-NT versions you might need to change AUTOEXEC.BAT to include SET directives)

4. Restart the GIMP and you should be up and running.

Check-cutters drop ball, bash Harvard, circle wagons; "consumerist" attitudes toward computing.

Paymaxx, a payroll services provider, recently confessed to a major
mistake that essentially made public many of their customers'
employees' W-2 forms. My firm uses Paymaxx to run payroll. So, as it
happens, does another Harvard-associated person's small computer firm.
This person, however, has more time (or more curiosity) than I, and
discovered a gaping hole in the system serving W-2 forms, a hole that
made it trivial to retrieve others' forms. This person did not create
the hole or “crack into” the system — just stumbled upon the hole left
open. What happened next was unfortunate.

The discoverer of the hole was in a bind; to confirm the existence
and nature of the hole, he necessarily performed some testing and
experiments. Upon forming a supported theory of the problem, he
contacted the company with his complaint, and a sales pitch for his
services to fix it. Was this morally correct? Certainly, he was
compelled to take action by knowledge that his security and privacy was
threatened; certainly, he was correct to inform the company. Certainly,
he was under no obligation to provide his expertise without
compensation. However, the quandary seems to center on the nature and
specificity of his notice / sales pitch to the company: did he wrongly
withhold information about the problem in a manner as to constitute
(morally, if not legally) a form of extortion?

The response of Paymaxx was less than satisfactory as well. In a letter to its customers, Paymaxx stated:

The hacker, is a 21 year-old Harvard student (or
graduate) with a history of similar stunts. He was a PowerPayroll
customer for nearly four years. In mid- February when we informed him
(and the rest of our customer base) of the availability of 2004 W-2
information on-line, he e-mailed one of our sales reps informing him
that he had found a flaw in the security aspects of our on-line W-2
application and that he would tell us about it if we would hire his
firm. We considered this a sales pitch and dismissed him.

The remainder of the letter is a bunch of hand-waving.
However, it is this paragraph that is most troubling. Why was their
customer referred to as a “21-year old Harvard student?” This seems to
me nothing more than an attempt to excuse their incompetence by
averring that it required an evil genius from Harvard (that spooky and
much-maligned ivory tower of mysterious egghead commies) to get into
their systems. Bad job, Paymaxx — there went your opportunity to own
up to your screw-up, be clear about how and why you screwed up, and
demonstrate the objective steps you've taken to prevent it in future.
Instead, you pled the Harvard defense, and tried to shift the blame
onto someone else. However, rather than inveigh against Paymaxx for
their wounded-animal response, I'd rather look to the systemic reasons
why we can expect this kind of problem throughout corporate America for
the forseeable future. I'll begin with a brief technical description,
and then give my theory on the attitude that leads to this kind of
result.

The problem was, schematically, that the URLs for retrieving W-2 forms were like this:

http://bogus.paymaxx.com/w2form?123456

Where, as you might guess, the next employee's form is 123457. This
is not exactly how the problem manifested, but it's close enough to
illustrate: the engineers who put that into play were either lazy or
stupid, not taking into account that changing digits in the URL is
trivial. Put in the right number, and you get the W-2 form, with name,
address, and earnings.

(Merely to demonstrate that I am not declaiming against their engineers
uninformedly, let me state that what needs to have been done is to 1.
use HTTPS, if they had not, and 2. engineer the sharing of a true,
non-trivially guessable secret (for example by snail-mailing a PIN to
each employee), and 3. putting a guess-number-count limit on the
retrieval dialog to prevent brute-force attacks. In defense of Paymaxx,
they are probably just the first payroll company to get caught with
something like this — I have chosen to stay with them despite, and
somewhat because of, their experience with this problem, since now they
should be more rightly paranoid about security and because I don't
expect any better from other firms.)

I can only speculate at the reasons behind this goof, but it does
fit with a general pattern I have witnessed, of what I term a “consumer
attitude” to data and computing. This attitude is promoted by the false
promises of the software industry to liberate us from the burdensome
task of comprehension — the notion that all software can be
“intuitive” and that humans and computers can interact without the
humans holding up their end of the bargain. Holding this attitude leads
to the implicit adoption of certain maxims;

  • All that is displayed visually (representation) is the thing itself
    (underlying form) and can only be manipulated thereby, and conversely,
  • How something can be manipulated via a visual interface is the only means of manipulating it.
  • (or, things work as they apparently do, and they don't work in other ways.)
  • The visual interface must permit a user with no or cursory
    training to access any conceivable functionality (by conceivable, I
    mean conceivable by a lay person with experience in the problem domain
    and describable in plain language, for example, “move the invoice date
    to the first Monday of the month;” I except functionality that lay
    persons would not think themselves qualified to describe, such as
    certain mathematical wrangling), and therefore,
  • Any program functionality that is reasonably described in plain
    layman's terms by someone familiar with the problem domain should be
    simple to implement, by a layman who is made familiar with computing
    tools (rather than by a programmer who is made familiar with the
    problem domain).

The attitude brings with it the conceit of thinking that others will
share the attitude — an assumption that always proves fatally flawed,
for even imagining a world devoid of legitimate curious “hackers,”
there will always be black-hat “crackers” who shun the maxims of
consumer attitudes in favor of experimenting, breaking things, and
seeking alternative scenarios. The consumer attitude is one of taking
the image on the screen at face value; of seeing the shiny parts of the
system as the important onces. It is also, unfortunately, the reigning
attitude in the business world, because having a “producer” orientation
to data and computing is hard and often unpleasant — much easier to
fire up Excel or Solitaire, than to write code! The consumer attitude
makes one believe that links are something clicked upon and not
manipulated, and dulls one to critical and proactive thinking about
security.

I
am not suggesting that every executive be intimately familiar with Web
application security before leading his company to make use of the Web,
but in the Paymaxx case, it apepars that even their engineers
manifested the consumer attitude, thinking shallowly about their
application's security.  Hiring these engineers, therefore was the big problem.  If executives have ONE imperative in their relationship to technology, it's responsible vendor selection! 

I suggest therefore that executives be made aware
of the existence of the consumer attitude and the problems with it, and
be trained to evaluate solutions and providers with an eye toward
avoiding “consumerist” technology thinking. Those who design, create,
manage, and maintain our technology infrastructure must have a
“producer's” attitude toward technology, understanding what the hard
problems are, and that they are hard, and not shying from depth of
understanding. Inevitably, this will grow to include executives at most
kinds of businesses, as all forms of organization rely increasingly on
information technology.

We are in a unique historical
moment with regard to this problem of attitude. The past century did
not suffer so greatly, for every shipping concern would naturally have
been managed by men who had sailed on ships, and every bridge-building
outfit would have been managed by engineers and architects — because
ship's officers and engineers had existed as professions for
generations. There might be one generation of management-age persons
who have a solid generalist background in computer science as of today,
and these few are a tiny fraction of the number needed to fill the
ranks of executive positions at IT-reliant firms. As a result, we are
stuck with dilettante consumers making critical decisions for
productive firms. Who would hire someone to oversee a pharmaceutical
plant's operations on the basis of his qualification of taking medicine
daily? It is absurd — but every time we put a “consumerist” person in
charge of an IT-reliant operation, we do the same thing.

There was a time when people did not hold a consumer attitude towards IT; indeed, the pendulum was too far in the other direction. People were scared witless about computers, and
they were seen as the domain of “wizards.” Indeed, secretaries became “pseudo-wizards” in their own right,
memorizing WordPerfect macros, and in effect writing their own programs
for routine tasks. This, of course, did not last: while some arcane jobs will always require engineers, for the
most part people got over their computer fears with training. 

It was accepted that to use a computer required
training and knowledge, as with using an automobile or a welder's
torch.  Then, with the rise of the Gog and Magog of Windows and
Macintosh, we found ourselves in the middle of an apocalyptic war
between two indistinguishable armies — meet the new boss, same as the
old boss. What they fought over was market share, but what they agreed
upon was promising the world that computers should be easy and
effortless.  Details of interface were the ideas in dispute,
rather than the underlying metaphors, attitudes, and concepts. And it
was amidst this battle — waged over the turf of the newly discovered
mass-market for computing — that the consumer attitude was
propagandized to the masses as well as the elites.

It made sense, too, in a world where computers were machines for
three families of applications: word processing and spreadsheets,
email, and custom (internal) applications. Word processing — at least
at a casual to moderate use level — is a great candidate for WYSIWYG,
know-nothing interfaces. Spreadsheets had the beautiful characteristic
of direct analog to well-understood ledger books and pocket
calculators, combined with a spatial orientation that paralleled the
WYSIWYG ideal of the word processor. Email was a finite
domain, and it had similar metaphors to familiar tools. And custom
applications, internal to a given organization, were the special
exceptions to the know-nothing rule — staffs were trained on workflow
processes, order entry “screens,” predefined queries written for a
particular purpose. Each internal application was like a special tool
inside the firm, usable for its one purpose, and only by those who were
trained.

And
how well this regime worked for a while! Get familiar with the clicking
and typing bits, and you've got the word processing, spreadsheet, and
email stuff down pat. Watch the training video or read the manual, and
you can use your company's order-tracking system or pull the
quarter-to-date sales figures from the Oracle database.  But what
happens as soon as Visual Basic for Applications is embedded in your
word processor?  What happens when your Excel model requires a
procedural language routine, or sources data from an external database?

If businesspeople are to operate effectively in the world of
computing, I believe that we must produce a thriving culture of rounded
generalist executives, interacting with honest vendors who make the problems
of computing as simple as possible — but no simpler! 
We must expect people to learn some of the underlying ideas behind the
abstractions; just as a freight forwarder must understand the underying
limitations and strengths of various forms of transport, regulations,
etc., an author of a complex data report must understand the
limitations and strengths of his data sources, the concept of the
normalization of data, timeliness and validity, etc.

Future directions: why a
consumerist “know-nothing,” and a technician, “specialized tool” model
are both insufficient ways for businesspeople to approach computing.
Necessity of generalist computing knowledge. Folly of having businesses
driven by IT run by modern-computing-illiterate executives (would one
run an oil company with no chemical engineers or geologists on the
management team?). Folly of expecting interfaces to require a constant
amount of learning (zero) while they expose a geometrically expanding
range of functionality to the user. Uniqueness of the generalist
computing skill set and how it is already as important to an executive
to understand data as it is to understand accounting and bookkeeping —
even if this is not accepted today.

FIX: Suppress Perl "uninitialized value" warnings (strongish medicine for clueful users only)

If you have written any Perl of moderate complexity, and especially if your Perl of moderate complexity has included CGI and Database interactions, (or any time you have to deal with undef or NULL vs. a blank string, and you might have either of the two), you have run across warnings like this (maybe to STDERR, or maybe in your /etc/httpd/logs/error_log):

Use of uninitialized value in concatenation (.) or string at ...

Use of uninitialized value in numeric gt (>) at ...

etc.  How can you stop these error messages (warnings, really) from blocking up your logs and your STDERR?

In fact, you should be somewhat concerned at your uninitialized value warnings.  After all, they are there for a reason.  A really good piece of code ought not to generate them, at least in theory.  However, sometimes you want the benefit of use strict and -w warnings, and you have at once good reason not to want to know about uninitialized values.  What might these be?

  • You are doing string interpolation into something where undef and "" are equivalent for your purposes (most web page work)
  • You are doing some conditionals or string comparisons based upon data that come in from one crufty source or another, like CGI, and you don't want to make a separate case for undef and "".
  • Relative quick-and-dirtiness where you want at least use strict in order to prevent egregious code but you don't need to hear about the semantic difference between undefined and the emtpy string.

In these cases, if you are using Perl 5.6+, you are in luck.  You can carefully wrap the specific section of code that has a good reason for not caring about undef values in a block (curly braces) and write:

{
  no warnings 'uninitialized';
  if ( CGI::param('name') ) {
    print "Hello, " . CGI::param('name');
  }
  else {
    print "Hi there.";
  }
}

Oregon Lottery Video Poker Specific Numbers

Executive Summary:

– I examine one Oregon Lottery video poker machine's reports over a circa three-year period.

– The “loss rate” for a brisk player at most games is $27 per hour, nearly four times the minimum wage.

– The “house edge” on most games is over five times worse than that with e.g. craps or blackjack.

– Players do react rationally though imperfectly to varying hold
percentages (house edge), and preferentially play games with a smaller
hold (smaller house edge).

– A bar with the legal limit of five such machines would net (EBIT)
nearly $39000 annually, with the state keeping the other $100,000.

– The lottery in effect pays $310 per square foot per year to rent bar
space, a 1000% premium over the downtown central business district.

– These numbers are below the statewide average, which is likely skewed by some very high-volume locations.

(Some numbers have been rounded to fit annual time frames.)

This machine had ten normal games, of which all had a 10% hold, except
one with 6% (“Flush Fever”) and one with 8% (“Oregon Gold”).  The
“draw high” game is 0% hold (no house edge).

Contrast this with ~5.3% for roulette, ~1.5% for craps, and ~1-2% for blackjack.

By far the most money was played on “Flush Fever,” the game with
the lowest hold.  This is probably because the difference between
a 10% hold and a 6% hold is so dramatic, that even without labeling, a
player can detect it readily.  About 45% of the action (money
played) was at this game. “Jacks or Better” got about 12% of the
action.  With 8% and 7% respectively, were “Deuces Wild” and
“Oregon Gold” (the 8% hold game). 

Since there are ten games, and the #1 and close #4 games are the least
and second-least holding games, we can surmise that people play more at the lower hold games.

Previously, I had speculated that the least you would expect a player to lose
under a 10% hold, playing one 25-cent game every ten seconds (brisk but
not blazing) would be $9 an hour. In fact, nearly all games recorded an
average bet of 75 cents or more.  That means that the hourly loss
rate would be at least $27 an hour, an hourly rate equating to a full-time salary of
$54,000 a year.

In fact, the overall hold percentage is reduced by two facts: 1.
players preferentially play the less-biased games, and 2. an
even-money, “double or nothing” bet with no house edge is offered to
winning hands.  The overall theoretical hold for the machine as
played should have been 6.37%, though it lagged slightly with only 5.8%
actually held.

Of the cash that had, over three years, been fed into the machine, more
than half is denominated in $20s (the largest).  About $267,000 in
bills had been put into the machine.  About $184,000 in winning
tickets had been printed.  Although on each coup the player might
expect 90% back, for every buck actually put in the machine, only 69
cents come back out.

The machine in question had about $1.4 million in action put through it
over 3 years (recall, action is calculated each bet, so it will be many
multiples of cash drop).  The total hold was about $83,000, about $2300 a month (one machine), or
about 5.8% of action (the “draw” game counts toward action, but cuts
down on the hold percentage since it has no inherent advantage).

How much play did this box get?  With an 83 cent average bet, and
476,400 dollars put through a year, that's about 574,000 bets per
year.  That's about 1752 a day, or 131 an hour through a 12-hour
day.   This sanity-checks my 360/hour estimate — 1/3 of the
time in rapid play seems sane.

The bar had five video lottery machines (the legal limit), but only one
of them was kind enough to tell us its financial history.  To
situate it, it's a youngish, dive-y 20s and 30s bar, with pool tables, the kind
of place where a cuba libre costs less than 4 bucks and they don't call
it a “cuba libre.”  In those terms, the patrons of that bar could
have had another 21,000 cubas libres over the last three years instead
of playing video poker.  This is not “el primo” territory for
video poker, though I would guess they do OK by video poker standards.

To do some quick math:
5 machines * 89000/year/machine = $445000 / year / bar drop
5 machines * 27667/year/machine = $138335 / year / bar hold
Retailer commission (average, per Oregon Lottery) 28% = $38733 / year / bar hold

To get $38733 annually, risk free, at 2% interest, you'd need
nearly a cool $2 million in the bank.  What does the retailer
stake for this?  About 25 square feet per machine, including chair
space.  With five machines, that's 125 square feet generating
$38733, or $310 / sq ft / year.  Today in Portland, Oregon's
largest city, you would be hard presesed to find Class A office space
renting for more than $30 / sq ft / year.  So the rental rate that
the Lottery is paying dive bars is only a 1000% premium over that for a
suite in Portland's toniest skyscraper.

To bring it back to earth, let's sanity check all of this against the known figures:

circa 2100 retailers * $138335 / year / retailer = $290 M hold overall

This is in sanity-range with the lottery's published $530 M figure
(there tend to be a few top-performers in the video lottery that skew
the results to the high end).

This could be an interesting case study for anyone looking at the
recently-again-in-the-news issue of Oregon's video lottery. 
Unfortunately, nobody is talking about how we can mitigate the harms;
instead, everybody just wants to wring more money out of the program.

INFO: What happens to ssh every 2:11:15?

I was getting a weird artifact in my logs.  A daemon process that was in charge of keeping an ssh connection open to a remote host was restarting ssh every two hours eleven minutes:

myuser 15208 0.0 0.0 0 0 Z 02:01 0:00 [ssh <defunct>]
myuser 15511 0.0 0.0 0 0 Z 04:12 0:00 [ssh <defunct>]
myuser 15548 0.0 0.0 0 0 Z 06:24 0:00 [ssh <defunct>]
myuser 15584 0.0 0.0 0 0 Z 08:35 0:00 [ssh <defunct>]
myuser 15619 0.0 0.3 3408 1704 S 10:46 0:00 ssh -T myhost ...

What the heck is going on? I was running this from behind a DSL modem, and I had experienced some intermittent problems with it before. Was it the modem? Googling on the model # indicated nothing similar reported by others. Was it my ISP or Telco? Phone calls to them indicated that 2 hours was the median time between dropped connections for some old modems, but not mine and not my circuit type. Hmm. Many people pointed to the TCP KeepAlive default of 7200 seconds — two hours — but my problem had a period of over two hours. Almost exactly, consistently, two hours eleven minutes.

As it turns out, the TCP KeepAlive time of 7200 seconds plus the default KeepAlive probe interval (75) times the default probe count (9) add up to 2:11:15.

If you want to change this for one reason or another, try:

echo "30" > /proc/sys/net/ipv4/tcp_keepalive_time

… or likewise (remember that you'll still have 11:15 worth of probe * count; lower those too if you need to know sooner). Better yet, read http://av.stanford.edu/books/tcpip/tcp_keep.htm for some actual theory on the subject.

One good use for this information is if you want to keep a persistent connection open between two machines using, e.g., Net::SSH::sshopen2 for a bidirectional remote connection to a process executed on a remote machine, but you're on a kind of flaky connection that can cause the connection to get dropped often but briefly, and the nature of the stuff you're doing is such that you want it to re-connect and try again rather than obliviously sit through the blip.

(The reason I ramble so lengthily on what particularly one might use this for is because you do NOT want to follow these directions if you're having a more common “momentarily flaky” connection sequela, such as you have terminal sessions that you wish to keep open despite a moment of flakiness — in that case, you do NOT want to enable short TCP keepalives, since they are really “detect deads,” and they will increase the likelihood that your blip in the connection will kill your terminal session.  In that case, you pretty much want to do the OPPPOSITE of this, excepting that 1. if you are behind a NAT router and your connection isn't actually flaky, you might really be seeing a timeout of the NAT table, not connection flakeage, and so you DO want to put a keepalive in shorter than the NAT table timeout [it's all a bit much, isn't it?] 2. you are probably best off just using “screen” and doing a screen -r to reconnect to an old screen when you get reconnected [screen is awesome for all sorts of reasons, and wth screen, if you can divorce yourself from the graphical burden, you've essentially got a total multitasking virtual desktop with persistent state as long as you've got a vt100 terminal].)

The way I would recommend would be the following:

1. Set up your local ssh_config to make sure you're using KeepAlive yes.

2. Set up your local tcp settings to have a short keepalive time and probe interval/count.  (Some kernels apparently don't behave with less than 90 seconds as the keepalive time but I have had success with much lower numbers.)

3. Set up your remote sshd_config to use the ClientAliveInterval and ClientAliveCountMax with reasonable values.  What this does is sort of a reverse and in-band version of what the TCP keepalive is doing on the local machine; the ssh daemon will send an encrypted signal across every ClientAliveInterval seconds and will hang up the connection if it misses CountMax of them in a row; this makes sure that the process you run on the remote machine gets hung up OK.

4. Make sure that your sshopen2 call and the sending and receiving of things along it recognizes when the SSH connection gets closed out and deals with it, such as by an eval loop and a reconnection in the event of $@ .

 

Oregon Lottery Video Poker Hold / EV statistics

This posting has three parts:

I. Why video lottery is a bad thing for the state to run
    in which I describe philosophical and practical problems with the status quo.
II. What the state doesn't tell about the mechanics of video lottery
    in which I describe the key statistical
metrics of how deleterious the game itself is, which metrics are not
published by the state.
III. My proposed scheme for remedying the situation.
    in which I indulge myself with presumed legislative fiat.

It will in turn be followed (preceded, for those of you reading in LIFO
order on the main page) by specific information hinted at in section II.

I. Why video lottery is a bad thing for the state to run.

Video poker (video lottery) as run in Oregon is deplorable for a number of reasons:

1. It uses a state-enforced monopoly to dole out “free money” to the private sector at the whim of the lottery administrators.
    A. No innovations, goods, or services are required
of lottery “retailers” — the machine is in effect an artificially
scarce rent-producing box.
    B. Unlike in a public-private collaboration such as
a utility, where the public (through the state) gives rights-of-way and
monopoly in exchange for the pulic good of universal power and
communications accessibility, the private portion of the video poker
cartel (bars) brings nothing to the table except access to drunk people
with cash.
    C. Giving free rents to qualifying and willing bars
in this manner dramatically disorts the economics of bar ownership,
subsidizing poor management of properties (it is not unusual that a bar should “break even”
on honest revenues, and make the annual profit from video lottery) and
penalizing bars whose morals or decorum make video lottery unacceptable.

2. It figuratively “addicts” the state budget to its revenues, giving itself institutional inertia.
    A. Video lottery sales comprise the lion's share of
lottery sales ($530 M / $895 M in FY 04), making the video lottery the
sine qua non of the entire operation.  (The portion of sales from
video lottery will rise dramatically as “line games” — slot machines
— are added.)
    B. The net transfers to the state in excess of $364
M  make for 3% of the state's budget.  This does not include
an effective “slush” fund of carry-forward earnings.  Legislators
must account for this gap if they take anti-lottery measures,
effectively hobbling forward-thinking lawmakers.  Job and
entitlement cuts that would follow budget cuts due to lottery rollback
give the lottery allies in the powerful public-employee unions.
    C. The lottery uses its earnings, before paying the
state its dividend, to pay for aggressive public-relations campaigns to
ensure its institutional lifespan.  If Republicans are afraid of
having to “starve” state-perpetuated “beasts,” this one is
truly fearsome.

3. It literally addicts gamblers, and constitutes a massive regressive wealth-transfer system.
    A. The video lottery is faster and more addictive
than paper lottery tickets; in fact, Oregon video lottery now
officially includes “line games,” or slot machines.  For more on
the addictiveness of the video lottery in South Carolina, see
http://slate.msn.com/id/36673/
    B. By reducing the need for a higher or more
progressive income tax, the video lottery subsidizes the highest-income
Oregonians.  (Not that we need to be paying particularly higher
income taxes, mind you, but there you have it.)

4. WORST OF ALL: Video lottery is the least transparently-treated part of the whole lottery scheme.
    A. The lottery commission emphasizes the paper
ticket games on its web site and its PR campaigns.  In fact, the
“ticket” lottery sales (excluding keno, multistate, and sports) amount
to less than 22% of sales — the vast majority comes from video lottery.
    B. The lottery publishes frequency charts and odds
of winning for the various ticket games, but publishes little about the
video lottery machines.
    C. Nowhere are the rules, EV / hold, and frequency audits of the video poker machines published. 

On the flip side, however, there are lots of things to like about the video lottery:

1. It gives money to the state budget.
[2. Some folks like to play it for fun.]

How can any principled person like the video lottery?  Religious
folks should hate it because it's gambling.  Liberals should hate it because it's regressive in
its redistribution and because some people have fun with it. 
Conservatives should hate it because it “feeds the beast” and teaches
fiscal irresponsibility, and because some people have fun with it. 
Libertarians should hate it because it's a state-violence-enforced
monopoly in what should be a competitive market and therefore people
aren't free to have fun with it on their own terms.  Humanists
should hate it for its fostering of antisocial behavior and its
reduction of human potential to negative-sum lever-pressing
domapinergic repetition.

II. What the state doesn't tell about the mechanics of video lottery.

What do I mean when I talk about “hold” or “EV?”  These are
measurements of the “edge” in a game (hold being the house edge), or,
from the player's perspective, the Expectation Value, given as portion
of amount wagered to be received back on each trial on average (note
that EV is typically stated less the initial unit bet, e.g. starting at
0, giving a positive or negative figure, -.05 instead of .95.  I
prefer EV given with 1.00 instead of 0 as a starting place, for ease in
the types of calculations I prefer.  If you are using the normal
method, simply subtract 1 from the EV numbers I use).

A quick primer on EV: for every $1 you bet, how much will you make
back, on average?  With a perfect coin-toss, it's $1 in, $1 out —
50/50 odds, or a 1.00 EV.  If the coin were biased to heads by 1%,
then your $1 bet on tails would win $0 [or “lose”] 50.5% of the time,
and win $2 only 49.5% of the time, giving you a 0.99 EV.  EV under
1 is considered “negative,” while the rare chance for a positive EV bet
is one in which the EV is greater than 1, also known as “getting the
best of it.”  Representative EVs include: roulette (American)
.947, blackjack ~.98, craps (pass line) .985, coin toss, 1.00, a year
in a savings account, ~1.01, a year
in the US equity markets, ~ 1.07.  Remember, EV is calculated with
each and every bet, like compound interest, so a .99 EV game done for
100 trials should net you back only 0.37 of your money (though in
reality variance messes up that neat figure).  Contrast that
with 100 years in the stock market at 7% [1.07 EV], which should give
you 867 times your money back, less taxes.  The total amount bet *
the number of bets is “action;” if you bet $5 100 times, you put $500
in action through, even though you may only have ever had $25 to play
with.  The “hold” is1-EV, or the percentage of each bet that the
house gets back; because it's calculated on each trial, the house
expects to earn the “hold” times the “action.”  Got it?

Poker players, stat jockeys, and others with a sense of EV will appreciate this chart from the Oregon Lottery's
annual report FY 2004:


Sales:
The Lottery had the highest sales
years ever in each of the following categories:
Total Lottery Sales:                   $895.18 million

Video Lottery Sales:                 $530.97 million


Total Traditional Sales:              $362.30 million

Keno Sales:                             
$116.48 million


Powerball Sales:                      
$  45.97
million


Sports Action Sales:                 $  10.00 million

At the bottom is Sports Action — a game that is technically beatable
(positive EV) if you are an expert handicapper.  Then comes
Powerball, a game that is rarely, but occasionally, positive EV because
it is progressive (if nobody wins, the jackpot can theoretically get
large enough to give a positive return).  Then comes Keno and
“traditional” — the stuff that most people think of when “lottery” is
said — all negative EV games but transparently so.  But almost
all the money, in reality, comes from video lottery — video poker (and
soon slots) — about which almost no information can be found.

Note that video poker can be positive EV — in a competitive market
like Las Vegas, where 1. operators have an incentive to reduce the hold
(increase the EV) and 2. progressive jackpots exist that, when unwon,
can boost the EV to positive.

Why, gentle reader, am I spouting off about the corrupt lottery system?

The answer is that I recently visited a local watering-hole with some
friends, and lined in the cramped ante-bar area were some video lottery
machines (no boon to those of us smooshed together, waiting in line to
get a drink).  My friend A. pointed to the screen of the nearest
one, which read

[Machine Configuration]      [Game Configuration]
[Reports]              
             [etc]

…instead of the usual blinky-beepy come-ons.  Curious, we
pressed the reports button, and found the machine happy to give us
printouts of the game holds, results, revenues, etc.  For your
edification, I will be posting this information shortly.  However,
in sum, please be aware of the following:

– The EV on most all Oregon video lottery games is 0.90, or put another
way, the “hold” is 10%.  This is a truly outrageous rate that
makes casino roulette (~5.3%) blackjack (~1-2%) and craps (~1.5%) look
like great deals.

– The EV on “flush fever” is 0.94, or the hold is 6%.  This is
very bad, but a hell of a lot better than others (like “jacks or
better”).

– The “double up” game, offered after each win, is even-money (EV 1.00).

(Keep in mind that all games played against a “house” with
near-infinite resources subject the player to “gambler's ruin,” a
situation in which the player underperfoms his EV because short-term
variances wipe out his bankroll.  This boost the effective hold
over the theoretical hold, in most cases.)

Why is this significant?  After all, everyone knows that the
lottery is a sucker bet, right?  Well, perhaps, but there are
sucker bets, and then there are sucker bets.  You are wiped out
exponentially faster with such a huge edge as 10%.  Consider a
brisk but realistic pace of 360 video poker hands per hour — the least
that may be wagered is 25 cents.  The total action per hour is $90
— at 1% hold, the player loses less than a buck.  But with Oregon
video lottery, the least he will expect to lose is $9 per hour — more
than the state's minimum wage.  Consider, too, that the average
bet is larger, and that the swings of the game put him at risk of
gambler's ruin.

The difference between a sucker bet, and a sucker bet, is huge. 

III. My proposed scheme for remedying the situation.

Here is what should be done:

[As a general premise: 0. “Indian” casinos should be replaced with a
brand-new way to take land and manpower and make jobs; I call these
special programs Indian “factories.”  The Indian factories can
make “goods” that are exported at a “profit.”  This will give them
jobs and tribal tax revenues.]

[UPDATE: The Wall Street Journal recently covered a development where
it appears others share the above premise — there is now a trend for
Indian tribes to use reservation land to build legitimate industry,
like concrete plants.]

1. The state gives up video lottery.
2. To combat grey market video lottery, to prevent folks driving out of
state to play other states' terminals, and to ensure proper income
reporting for taxation, the state forms a video lottery audit board.
3. The audit board licenses establishments to have a certain number of
machines present, and licenses machine owners to run any number of
privately-owned machines with publicly-audited software.  Each
machine's hold percentage may be set and changed by its owner.
4. Each machine must be conspicuously labelled with its hold percentage / EV (see II. above) and most recent audit.
5. The audit board charges fees sufficient to fill its budget. 
Overage is rebated to license-holders, preventing the state from having
an interest in promoting video lottery.
6. Machine-owner licensees contract at arm's length, at-will, with
establishment-licensees as to machine placement.  This creates a
healthy, adversarial competition for the customer's dollar between the
booze and the slot machine.
7. The state makes its money off of income taxes.

The ends my plan serves are:
1. Elimination of state dependence upon, promotion of, or interest in, people losing more money at the video lottery.
2. Reduction in the “hold” and profitability of video machines by market forces, thereby:
3. Reducing incentives for “Indian” casinos by taking pricing power away from them on video poker-type games.
4. Reducing incentives for poorly managed bars by lowering the amount of the received subsidy from the video lottery.
5. Maximizing  happiness by promoting more play per unit wager for
lovers of video lottery and eliminating video lottery from some bars
for whom the lowered margins do not justify keeping the games.

FIX: Evil MSN Search spyware behavior for DNS errors in MSIE

In Microsoft Internet Explorer >= 5, a DNS error (asking for a hostname that doesn't exist) causes IE to pull an evil stunt and feed the requested URL to http://search.msn.com/dnserror.aspx , where it is used to search MSN and logged for who knows what nefarious purpose.  Rather than being opt-in behavior, this is opt-out, and rather than making it clear with a choice in options like “Automatically search when a site is not found”, opting out is a matter of making this choice:

Tools:Internet Options:Advanced:Search from the Address bar:Do not search from the Address bar.

Irksome.