March 23, 2010

Heh - a 'certified' database administrator

Great story about some maroon who took a class and got himself certified. From The Daily WTF:
The Certified DBA
“I’m not questioning your expertise,” Paul cautiously said to the Certified DBA, “it’s just that I’m just not used to requests with… this level of detail.”

Paul should have done what he was asked, exactly how he was asked to do it. After all, he was not an expert but just a lowly systems administrator. Fortunately, the Certified DBA made sure to keep him in his place.

“I would expect not,” smirked the Certified DBA, “system administrators generally don’t know how to tune for database performance, so I have to be very specific on the server configuration.”

The “configuration” that the Certified DBA was referring to was a rather complex set-up of disk partitions and RAID. Per his request, the server Paul was to set up should have six disks, each with ten partitions set-up. Corresponding partitions on disks 1, 3, and 5 were to be concatenated (not striped) to each other via RAID, while the partitions on disks 2, 4, and 6 served as mirrors of their corresponding partitions on disks 1, 2, and 5. All of these partitions were then to be mounted as directories (/p1, /p2, etc) in the file system.

“Fair enough,” Paul acquiesced, “but I’d like to understand why this helps with database tuning . I mean, wouldn’t this just create slower reads and writes? I just don’t see how it helps perfor—”

“I’ve been doing this for fifteen years and it’s not like you can understand database tuning overnight.” The Certified DBA was getting aggravated. “The long and the short of it is: the partitions allow me to place datafiles on different physical parts of the disk. The outer edge of the disk is the fastest, so I put indexes in partition 1. Large binary data goes on partition 10. And so on.”
The database, of course, runs very slowly. There is a fix from the DBA but then, the DBA goes on vacation:
Paul, being the lowly system administrator, was not in a position to argue with the Certified DBA, and had little choice but to do what he was told. But instead of doing just that, Paul decided to try something different.

While the Certified DBA was on vacation, Paul took a pair of the 10,000 RPM drives, mirrored them, and then set-up a series of directories (/p1, /p2) on the file system. When the system was brought back online – using a single drive and a single partition – the I/O reads went down by a couple orders of magnitude. The overall application performance improved dramatically, and the users were appreciative.

But there was one thing, however, that was missing: approval by the Certified DBA. Undoubtedly, he’d be upset that his instructions weren’t followed, and accuse the lowly sysadmin of subterfuge and insubordination.

With the Certified DBA returning the following day, Paul was desperate and changed the “df” command such that it executed a shell script that looked identical to the real df command: it listed ten partitions, each mounted in /p1, /p2, and so on.

And the shell script worked. In fact, it worked so well that the Certified DBA took a screenshot of it and used it in an article he wrote about database “tuning” for a trade magazine. And, being the nice guy that he was, the Certified DBA gave Paul a signed copy of that issue.

Though the Certified DBA is long gone, Paul still keeps the signed magazine hanging in his office. It’s taught him more about database tuning than he ever could have hoped for.
When I worked at Microsoft, I was a member of the lab heardware support team for their SQL Server database and then I moved over to the Enterprise Scalability Lab where I got to play with some serious iron. Tuning an array to optimize it for a database is not a black art. Tuning the database software is but the iron is pretty straightforward. Machines that I administered and helped to set up were used in several TPC-C benchmarks... Posted by DaveH at March 23, 2010 3:01 PM | TrackBack
Comments
Post a comment









Remember personal info?