09.17.07
Posted in Technology, Linux at 5:27 pm by Stoner
Multicast in Linux is a GREAT BIG BLACK HOLE. There is no suitable documentation on it. What I’ve found is many years old, for the 2.0/2.2/2.4 kernel tree or in reference to getting Linux to route multicast packets. The best doc I’ve found is by Juan-Mariano de Goyeneche - from 1998!
Additionally, there are little to no user-space tools for working with multicast. You can use ‘netstat’ to see what multicast groups the host is subscribed to and the number of subscribers…other than that, you are powerless to do anything. What would be nice would be tools to:
- list the subscribers (by process ID and socket number (since an app can subscribe to multiple multicast groups)
- detach a subscriber from a group, something akin to “kill -9 PID:socket_number”
- see the traffic stats for each multicast subscription
Linux is definitely lacking in the multicast department.
Permalink
Posted in Life, Family, Hobbies at 12:53 pm by Stoner
A while ago, I purchased a replica 1851 .44 caliber Navy revolver - a black powder, cap-n-ball revolver. I finally got to shoot it over the Labor Day weekend with my father. It was a pleasurable experience and it gave me a real admiration for the soldiers and officers who fought in the American Civil War. The effort that goes into loading and firing a cap-n-ball revolver (or rifle even) is tremendous!
A good soldier, with a reliable rifle, was able to load and fire 3 rounds in a minute. Considering how long it took me to load 6 chambers of my revolver using a pre-formed charge (no measuring required,) ball and wadding, then to put the caps on the nipples and finally to fire the weapon - I would have been counted among one of the dead. Of course, I experienced several misfires where the cap would go off but it wouldn’t ignite the powder in the chamber.
On the plus-side, out of 12 chambers, I managed to hit a target (8″ in diameter) 4 times at 25 yards, including 2 in the center bullseye. Considering it was my first time shooting that revolver and my first time shooting a cap-n-ball, I’d say it wasn’t too shabby.
Permalink
09.03.07
Posted in Life, Family at 9:25 pm by Stoner
This weekend was great. Absolutely great. Spent a lot of time with my brothers and my parents. I drove to my parents last Friday night, woke up and helped pour cement with my father, my middle brother and some neighbors. Now, pouring cement, in and of itself, sucks. But the time I spent with my brother and father was priceless. My brother taught me how to properly mix cement (ratio and consistency) while helping to finish off my parents’ basement after almost 40 years was nice.
After pouring cement, we went to my other brother’s place near the Pennsylvania/Ohio border. I always enjoy spending time with my nieces and nephew and seeing their new house was nice, too.
After that, on Monday, my father and I went to the sportsmen’s club to shoot. I had my Springfield 1911 and 1851 Navy black powder revolver (reproduction, of course) - he had his 1911 (I forget the manufacturer.) We squeezed off a bunch of clips from both 1911’s and fired several cylinders of the Navy. I actually hit the target 4 times out of 12 shots at 25 yards - not bad considering how inaccurate cap-n-ball revolvers were and it was my first time shooting it.
My dad enjoyed the shooting outage and that pleased me greatly. My father just retired…which to me means we don’t have much time left together, 20 years nominal. If that sounds like a long time, consider this: we were cleaning up after pouring cement when I noticed a young woman across the street. I made a comment and my father said “that’s their daughter, she’s 16.” I was stunned because the last time I had seen her, I was twice as tall as her and she was just starting kindergarten. That made me realize just how fast time can fly if you let it.
Well, I don’t want to turn around and see my father lying on his death bed and suddenly realize that I’d wasted all that time. I want to learn from him, spend time with him, do whatever I can to make his retirement happy and fulfilling. And that goes the same with my mother - although she doesn’t like to go shooting. 
Permalink
08.17.07
Posted in Technology, MySQL at 2:32 pm by Stoner
MySQL has a unique feature whereby you can select the storage engine for each table. If you don’t know what a storage engine is, it is responsible for managing the physical data on disk - from data files and indexes to the memory buffers used to access the data and the locking needed to prevent multiple people from clobbering each other’s changes. This is both a boon and a bane for Systems Administrators and DBAs.
It’s a great feature because developers, Sys Admins and DBAs can select the engine with the properties they require. For example, the InnoDB engine supports ACID-compliant transactions, foreign keys and row-level locking. The MyISAM engine, while being lightning fast on reads and supporting concurrent inserts, does not have ACID-compliant transactions or foreign keys and implements table-level locking. Other engines are available, such as memory-based tables, clustered tables, federated tables (reference a table on server B from server A) and more. This level of flexibility can’t be found in any other database server.
This flexibility comes with a stiff price, though. If you mix engines in a single database then you must be aware of several limitations:
- Foreign keys support in the InnoDB engine does not extend to other engines, meaning you can’t have a InnoDB-based table define a foreign key on a MyISAM-based table.
- Likewise, transaction support in the InnoDB engine does not extend to tables defined using another engine.
- Some buffers are engine-specific so you may need more memory than if you used a single engine for all tables.
Permalink
07.31.07
Posted in Uncategorized at 5:27 pm by Stoner
Was at MySQL training all last week. Good stuff. Sarah rocks. I’m trying to apply what I know to existing client installations. My biggest customer has some really ugly SQL statements. They frighten me. I downloaded Visual Explain, which really helps identify where in a SQL statement things may be choking.
Why do cats seek out cloth surfaces (carpet, sofas, recliners, shoes, etc.) to hack hairballs onto?
How can we, as a society, advance when grown men, some who have been on this Earth for more than half a century, cannot piss into a urinal without spilling onto the floor? It’s not that hard, guys. Grow up.
Permalink
07.13.07
Posted in Technology, Life, MySQL at 12:04 pm by Stoner
Ok…I’ve been crazy busy with job and home stuff. Doesn’t mean I stop thinking about stuff.
- Do we really need another text editor/Personal Information Manager/MP3 player? I monitor Gnomefiles and Portable Freeware RSS feeds and I’m constantly seeing these types of apps being created, promising to be a better than anything out there with whiz-bang features not seen anywhere else. Got news for you, you aint showing us nuttin’ new.
- The First Bank of Delaware needs to DIAF for offering loans at 99.25% interest. That’s right, 99.25%! It’s done through their CashCall marketing arm. Yes. The one offered up by has-been Gary Coleman. They’re preying on people who can’t manage their money.
- MySQL-Proxy is damn cool.
- MySQL-Proxy combined with memcached rocks da bomb!
- Managing hundreds of servers with Microsoft Office documents and Microsoft Sharepoint portal totally sucks nuts. I’d rather slash my wrists with a dull blade, then pour salt in the wounds.
Permalink
06.20.07
Posted in Technology, Life, MySQL at 11:09 pm by Stoner
Been a while since I wrote anything…not for lack of anything to write, just real busy. I’m finding myself having to force myself away from the computer to maintain my sanity.
On the home front, I’m going to be installing a ceiling fan in the only bedroom that doesn’t have one. I had to go buy a metal junction box because all the boxes in this house are the blue plastic ones. That’s fine and dandy but I’m not going to hang a ceiling fan from one. Don’t know if I’ll need it but I also bought a brace so I can mount the box to a rafter if it isn’t close enough to one already. After that, I’m going to start ripping down the ceiling in the basement in preparation for remodeling it. It’ll be a nice mess.
Then I’m off to California for a week - business related. I fly out Sunday morning at 7:30 and fly back Saturday.
Attended a MySQL webinar (man, do I hate that word) on replication and scaling out MySQL. I’m very familiar with replication but wanted to see what the MySQL folks are up to. Also got approval (and advance payment!!!) for MySQL training in Washington, D.C. in July. I don’t have to fork over a couple grand and submit an expense report after - which is nice.
Someone implemented an idea I had been working on - namely backing up MySQL binary logs on a MySQL replication master server. I had a shell of such a script written in Perl - this guy did it in bash. Six of one/half a dozen of the other - I can just reuse his code and not reinvent the wheel. Open Source is beautiful that way.
Found two scripts for making practical sense of the MySQL “show status” command. Problem is, they generate flat, unparsable text and don’t retain the info for later trending. Pity. I’ve started work on that project so hopefully I can start to get a better understanding of how our MySQL databases are performing over time.
Permalink
06.09.07
Posted in Uncategorized, Technology at 3:01 am by Stoner
It seems that every day, Microsoft gives stronger and stronger evidence that they are bad for business. Bug and security holes in products aside, they are professional double-talkers. They either outright lie or say one thing then spew something else shortly after. It’s disgusting. Here’s a couple examples.
Microsoft has, for a while, been saying that Linux and other Open Source Software projects infringe on many software patents. They kept building up, hinting about possible lawsuits, etc. Then they finally broke out and stated that Linux and OSS violates 235 of their patents. They even break them out into general categories like email-related or GUI-related. But, when pressed for details, Microsoft claimed they couldn’t specify which patents because it would be too cumbersome, administratively, to collect that information.
Umm. Hello? If you counted them, then you already collected them into some tabular format. Otherwise, how could you count them? This is the type of bullshit business practices I’m talking about. Here’s another one.
Jamie Cansdale, of London, was awarded a Microsoft Most Valuable Professional award for his work on a product that allows developers to better test the program code they write. The product integrated into Microsoft’s Visual Studio product and was available for all versions of Visual Studio. Later, Microsoft started threatening him with lawsuits because they didn’t want his product to be used with their “Express” version.
What the hell kind of message is that? “We’re awarding you one of our most prestigious honors. Oh, and by the way, we going to sue you for making it available for all versions of our product.”
So, ask yourself. Is this the type of company I want to do business with?
Permalink
05.26.07
Posted in Technology at 5:04 pm by Stoner
MySQL is a great database. Packed with tons of features. Very easy to set up and get running. Easy to maintain. But it does have some serious drawbacks if you want to set up enterprise databases. I’m presenting my opinions about the MySQL 5.0 release of the database, based on my direct experiences with managing several medium-sized databases
Immature stored procedures
Beginning with MySQL 5, developers and DBAs now have access to triggers and stored procedures. These two key pieces of technology have been in other databases for years where they’ve had time to mature and grow with the product. MySQL’s implementation hasn’t and there will be growing pains to go along with it.
Backup and restore
Obviously, backing up and restoring a database is a critical business continuity function. The mysqldump utility is very flexible when it comes to backing up a MySQL database. You can be very picky when it comes to the database, the tables it backs up, even using a WHERE clause to get more fine-grained with the data. It generates straight SQL statements with some special MySQL extensions enbedded in comments. To restore, you simply pipe the dump file into the mysql client program and it’ll execute the SQL statements in order and rebuild the database/tables/rows. There are a couple problems with this approach:
- If you use the MyISAM storage engine you may encounter what I call “the back up lock up.” The MyISAM storage engine only supports locking at the table level. This means that when you make a backup, you lock all the tables for read access (so the dump is consistent across all tables being backed up) while mysqldump does it’s thing. This means your application could come to somewhat of a halt, depending on how well it handles not being able to write to the database or prolonged pauses. You can, of course, choose to not lock the tables, but then you have no guarantees that the data across tables is consistent. Fortunately, mysqldump is very fast. For small databases (a couple hundred megs or smaller) mysqldump will lock the tables for a minute or less. For a 5 gig database, expect to be locked for 5-10 minutes.
- Now that you have a backup, you may need to restore it from time to time, either on a developer/QA box to test new code, to verify the backup/restore process works (you *are* doing this, aren’t you?) or because a hard drive crashed and you need to restore the data on the production database. Again, the storage engine comes in to play. MyISAM tables can be written to very, very fast and a restore of hundreds of megs worth of data can be done in a matter of minutes. The InnoDB engine, with its transaction and foreign key support, takes quite a bit longer. A 5 gig dump file will take hours to restore.
The commercial database players have more sophisticated means of performing backups and restores that don’t suffer from these issues.
Relatively few who really understand how MySQL works
With the “big boys” in the database arena, there are a good number of highly qualified DBAs out there who know how to run the system and make it purr like a kitten. There are certification programs, conferences, books, people who “grew up” with particular databases, etc. With MySQL, you’ll find a lot of folks like myself - admins or developers who know how to set up and the basics of MySQL, but aren’t real DBAs. We got that title out of necessity (no one else available to do it.) For true enterprise implementations, you need someone who eats, sleeps, breathes MySQL 36 hours a day. MySQL AB does offer training so it’s only a matter of time before there is a large pool of MySQL DBAs out there.
Replication works to a point
MySQL has a simplistic replication feature that allows you to easily maintain copies of data on multiple database systems. In fact, its commonly used to build a “cluster” of read-only databases that an application queries from. The app will need to write to the master database, of course, but this gives you the ability to scale out your capacity to query data. The problem with replication is, when you have a database that gets a lot of INSERT/UPDATE/DELETE queries, you can find yourself in “replication lag,” where the replication slave databases fall behind the master. This happens because replication only uses 2 threads: one to read the data from the master and one to write it to the local data files. On the master, there are several threads writing to the data files but on the slave, there is only one (because replication works at a different level than regular database connections.) More processors won’t help, only faster processors and faster disk drives. You’re only other option is to replicate only a portion of the data but that may defeat the purpose.
Can MySQL be used on the enterprise? Absolutely. Should it be used in the enterprise? Absolutely. You need to examine where and how it will be used and plan accordingly. There are many big name companies out there using MySQL to run business-critical applications (Google uses it to run their company financial systems, Yahoo uses it a lot behind it’s web sites.)
You will get out of MySQL what you put into it. If you spend the money for a quality hardware platform and hire a high quality DBA, you can go far with MySQL. If you put it on generic hardware with little to no redundancy and let your overtaxed IT department (or a developer) manage it, it’s going to be a pain point for you. MySQL AB, the company behind MySQL, offers a commercial license and top notch support. While you’ll never “go it alone” due to the large community around MySQL (there are quite a few forums and web sites dedicated to MySQL out there,) having a phone number to call can be quite a time saver. And MySQL developers typically do the support for the commercial licensees, so it doesn’t get much better than that.
Permalink
05.24.07
Posted in Technology, Funny Stuff at 11:06 pm by Stoner
All Sys Admins start their day by reciting this prayer:
Our Server,
who art in the data center,
hallowed by thy CPU.
Thy computations done.
Thy process ended.
With results written to disk
and to tape.
Give this day our daily reports,
and give us not confusing errors,
but produce graphs and charts
for the PHBs I must endure.
Leads us not to core dumps,
but deliver us from lusers.
For thine is the Power Supply,
the RAM and Hard Disk, forever.
Amen.
Permalink
« Previous entries · Next entries »
