I have a bulk import process that needs running nightly. Currently there are around 4,200 "rows" to process, which actually can encompass many tables so row is not entirely appropriate. The problem is that the script poops out after ~200 "rows" with memory limit errors. While increasing the memory limit is do-able, I am not interested in that as a solution currently. First, I recorded some numbers to benchmark where the script was in memory consumption. The first number is the "row", the peak is how much memory the script has allocated, and the mem is the delta in memory allocation per row.

0 peak: 41,103,408 mem: 29,884,416 1 peak: 42,440,264 mem: 1,310,720 2 peak: 43,613,848 mem: 1,310,720 3 peak: 43,893,960 mem: 262,144 4 peak: 44,223,040 mem: 262,144 5 peak: 44,896,296 mem: 786,432 6 peak: 45,671,560 mem: 786,432 7 peak: 45,865,952 mem: 0 8 peak: 46,418,272 mem: 786,432 9 peak: 47,917,888 mem: 1,310,720 10 peak: 48,566,312 mem: 786,432

After the initialization phase, memory allocation increases fairly steadily and it is not long until the 128M memory limit is reached. This is unacceptable as I know some "rows" should be much closer to 0 as nothing is imported on the majority of rows. My first solution was to disable logging:

sfConfig::set('sf_logging_enabled', FALSE);

The initial memory allocation was decreased, but the running deltas remained higher than expected. Second, I inserted a ton of unset() calls in the various functions. This dropped my deltas a little:

0 peak: 30,611,472 mem: 20,971,520 1 peak: 31,969,760 mem: 1,310,720 2 peak: 32,180,112 mem: 262,144 3 peak: 32,333,216 mem: 262,144 4 peak: 32,525,144 mem: 262,144 5 peak: 32,656,616 mem: 0 6 peak: 32,863,736 mem: 262,144 7 peak: 33,103,264 mem: 262,144 8 peak: 33,455,544 mem: 262,144 9 peak: 33,754,288 mem: 262,144 10 peak: 33,984,976 mem: 262,144

But allocation still killed the import before it could complete. Browsing through various sites, I discovered Propel had a hard time cleaning up circular references, which meant PHP couldn't garbage-collect that memory. However, to combat this, Propel 1.3 offers a static method disableInstancePooling that allowed me to override Propel's desire to keep instances around. Adding:

Propel::disableInstancePooling();

to the beginning of the import gave me these results:

0 peak: 26,569,632 mem: 17,301,504 1 peak: 28,582,152 mem: 2,097,152 2 peak: 30,455,352 mem: 1,835,008 3 peak: 30,536,176 mem: 0 4 peak: 30,536,176 mem: 0 5 peak: 31,517,088 mem: 1,048,576 6 peak: 31,534,152 mem: 0 7 peak: 31,552,120 mem: 0 8 peak: 31,589,632 mem: 0 9 peak: 31,695,504 mem: 0 10 peak: 31,695,504 mem: 0

Now new memory was allocating only when the import was actually doing something of significance. In fact, watching the import proceed with the deltas displayed, I could observe the memory decreasing at times, prolonging the life of the script by orders of magnitude. Whereas before the script was processing ~200 "rows", it currently processes the whole batch (4,237 "rows" currently) in one go. As the importable "rows" increase, I know I won't be butting up against memory limits for some time.

A letter to Congressman Goodlatte, concerning the invasive way the Commerce Clause is invoked to pry into the private lives of Americans.

Congressman Goodlatte, I would like to bring to your attention the cases of Wickard v Filburn (http://en.wikipedia.org/wiki/Wickard_v._Filburn ) and Gonzales v Raich (http://en.wikipedia.org/wiki/Gonzales_v._Raich), wherein the Supreme Court found "that Congress can regulate purely INTRASTATE activity that is not itself “commercial,” in that it is not produced for sale, if it concludes that failure to regulate that class of activity would undercut the regulation of the interstate market in that commodity" (emphasis mine). Congress has seemingly confused economics with commerce. Setting prices is not Congress' job. Setting quotas, to be met or not exceeded, is not Congress' job. Telling an American what they can and cannot produce, and in what quantities, for personal use, is not the job of Congress! Taking the Filburn case further, one can see it being applied to any number of self-sufficiency initiatives one may follow, like home gardening or personal solar/wind/hydro power. As we can see with HR 875 and S 425, regulating home gardening is already being proposed. I am glad to see that no Republicans are supporting it, but 41 Democrats are (HR 875, that is). As home power generation technology becomes more affordable and effective at displacing power drawn off the grid, how long until power companies join together to have Congress regulate how much energy a person can generate locally? Congressman, this trend worries me as I'm hoping it worries you. We need strongly worded legislation that reigns in Congress' ability to use the Commerce Clause as carte blanche to stick it's nose in the economy. Congress has the power to regulate commerce between the Several States, NOT in the private lives of Americans, no matter how much Congress "thinks" it may affect the national economy. Please respond by letting me know how you are personally going to reign in Congress' power (something any true Republican would actually do, even though it reigns in their own "power"). I would like to work with you on drafting this legislation so that it protects the individual's ability to take care of themselves and their families without Congress influencing those decisions. Sincerely, James Aimonetti

If you have gotten the above error in your syslog, you probably are not able to connect to wireless APs that require WPA passphrases, but can connect to ones that require WEP or no security. While searching for solutions, there were some that offered solutions for controlling the power of the card (for temperature-related issues), but those did not address my issue. Fortunately, the solution was fairly simple for my problem. I have Ubuntu Jaunty, with the 2.6.28-15 kernel. The resolution came when I looked in synaptic at the installed linux-restricted-modules and found I had two instances from older kernels still installed. Purging those completely from the system, and retrying to connect to my WPA2-enabled AP succeeded. Give it a shot.

This is the letter I sent our Representative concerning Social Security. I only write to get insincere, pandering, and uninspired form letters in the mail. I like mail.

Rep. Goodlatte, It is time to allow younger Americans to opt out of Social Security. I need that 12.4 percent of my income more than the Federal government. With the latest report from the Trustees Report, Social Security will be insolvent by 2016, perhaps sooner if economic conditions do not improve. In seven years, I will be in my mid-30's; hardly the age to begin drawing on it. I object to being forced into Social Security for many reasons: 1. I disagree with the income tax in general. My labor being traded for fiat currency is not income. Having to work four to six months just to pay taxes makes me a slave, or indentured servant at best, to the Federal Government, and I am pretty sure we have an amendment that specifically touches on that subject.

1. The tax revenues have been used to cover the deficits in the rest

of the government's budgets, and the only way to make the system sustainable is to give the participants a negative rate of return on their money. If I contribute $100,000 over my career to Social Security, I lose in at least two ways. First, inflation steals the "value" of the fiat currency. We could discuss at length the lack of real value (gold- and silver-backed currencies), but for now, even the fiat currency will lose value over the next 50 years (assuming the dollar doesn't implode). Second, if I do not live long enough to use the $100,000 I put in, I have lost some percentage of that money. I would rather have it now to invest as I see fit, rather than hope I live long enough to spend the money I earn! There are those much smarter than me that have realized the Ponzi nature of the Social Security scam. What is the difference between Social Security and Bernie Madoff? Governmental versus private scams are still scams and yet Madoff is going to jail and the administrators of Social Security are not. Bob, it is time I had a way to get out of Social Security. Let me be a responsible adult and care for my family's financial security. I have no confidence in the Federal Government's abilities to administer social programs, and I despise being forced into under force or threat of force from the IRS. By the way, taking of property by force or threat of force is theft, illegal by law, and yet the Federal Government is allowed to do exactly that on many occasions. So, I implore you, fight the good fight and give me and my peers the ability to leave the Ponzi scheme having only lost a small portion of my lifetime wages. Eagerly looking forward to a form letter or email telling me you'll look into it without committing to anything useful! Sincerely, James Aimonetti PS - For your reference, in case you or your minions want to learn about the mess that is Social Security, read http://mises.org/story/3469.

I recently needed to move static content from a live site to a cachefly account. Rather than go through the directories, looking for the resources (js/css/images) I needed to ftp, I thought, "Man, this sure sounds like it could be automated". The first step was to collect a list of items that needed ftping to cachefly. I know what you're saying, "Use find!" In case Ben is reading this, find "searchs for files in a directory hierarchy" (that's from find's man page Ben). I wanted to separate the resources out so I ran three different invocations. For javascripts and css, the invocation was nearly identical:

find . -name '*.js' > js.libs
find . -name '*.css' > css.libs

Images were a little trickier. Most of the images are static content, but some are user-generated, likely to change or be removed. These do not go up to the CDN (at least for now). The user-generated content is located under one directory (call it /images/usergen), so we simply need to exclude it from find's search.

find -path '*images/usergen*' -prune -o -path . -iname '*.gif' -o -iname '*.jpg' -o -iname '*.png' > image.files

The important parts:

• #+BEGIN_EXAMPLE -path 'images/usergen' -prune #+END_EXAMPLE

  Remove any found items that contain images/usergen in the path name.

• #+BEGIN_EXAMPLE -o -path . #+END_EXAMPLE

  Search within the current directory (the root of the project).

• #+BEGIN_EXAMPLE -iname '.gif' -o -iname '.jpg' -o -iname '*.png' #+END_EXAMPLE

  Match, case-insensitive (-iname instead of -name), any files ending in gif, jpg, or png.

We are then left with three files, each line of which contains the path, relative to the project root, of each resource I want to upload. I created a simple php script to upload the images, maintaining the pathing, to cachefly. So an image with relative path /images/header/header\_left.png would now be accessible at instance.cachefly.com/images/header/header\_left.png. So the images are now up on the CDN. Now we need our code to point there as well. Fortunately, most of the resources were prepended with a domain (stored in the global $live\_site). So the src attribute of an image, for instance, would be src="< ?= $live\_site ?>/images/header/header\_left.png". Creating a $cachefly\_site global, we now only need to find lines in our code that have a basic layout of "stuff……$live\_{site…stuff…..png}" where stuff is (.*) in regex land. So we utilize two commands, find and egrep. Find locates files we want and egrep searches the found files for a regex that would locate the resources in the code. So first, we build the regex. We know a couple elements that need to be present, and one that should not be present. Needed are live\_site and a resource extension (js/css/jpg/png/gif), and not needed is the "images/usergen" path, as this points to user generated content. So the regex becomes:

'live_site([^images/usergen])+.+(png|gif|jpg|css|js)'

This is the arg for egrep (the -l switch means print the file names that have a match, rather than the lines of a file that match):

egrep -lr 'live_site([^images/usergen])+.+(png|gif|jpg|css|js)'

Now we need to tell egrep what files to search using find:

find . -name "*.php" -exec egrep -lr 'live_site([^images/usergen])+.+(png|gif|jpg|css|js)' {} \;

We then store this list of files into a shell variable:

export FILES=`find . -name "*.php" -exec egrep -lr 'live_site([^images/usergen])+.+(png|gif|jpg|css|js)' {} \;`

Now that we have the files we need, we can search and replace $live\_site with $cachefly\_site for resources. The goto command for search and replace is sed. The sed command will look generically like this:

sed -i 's/search/replace/g' FILE

We actually have two issues though. Due to the nature of the code, we have to account for the $live\_site variable being passed in via the global keyword. So not only are we searching for resource files, but we also have to add $cachefly\_site to the global lines to make sure $cachefly\_site is defined within the function where output is generated. Searching and replacing resource files is pretty easy:

sed -i '/live_site.+\|js\|css\|gif\|png\|jpg/s/live_site/cachefly_site/g' $FILES

$FILES, of course, came from our find/egrep call earlier. There is one catch to the regex used here. It is actually of a different generic form than mentioned above:

sed -i '/contains/s/search/replace/g' FILE

With this format, we put a condition on whether to replace text, meaning the regex in the "contains" portion must be matched before the search and replace is performed on that line. So our sed above says if the line contains live\_site, followed by anything, ending in one of the listed resources (\| means OR), then replace live\_site with cachefly\_stite. I left of the $ since its common to both variables. Running the sed command replaces everything nicely, but when we reload the page, we see notices about $live\_site being undefined and resources being pulled from the host and not cachefly. So we need to handle the global importing. This one is a little tricker because we are not really replacing live\_site with cachefly\_site, but appending it to the list of imported globals. So a line like

global $foo, $bar, $live_site, $baz;

becomes

global $foo, $bar, $live_site, $cachefly_site, $baz;

The other trick is that the global line should not already contain $cachefly\_site. We don't need that redundancy. So, without further ado, the sed:

sed -i '/global.*live_site.*\(cachefly_site\)\{0\}/s/live_site/live_site,\$cachefly_site/g' $FILES

The "contains" portion matches the keyword global, followed by stuff, followed by live\_site followed by stuff, with cachefly\_site appearing exactly 0 times (denoted by \{0\}). This ensures we only replace live\_site when cachefly\_site is not in the line already. The "search" portion is easy; search for live\_site. The replace portion replaces live\_site with live\_site,$cachefly\_site. This takes into account when live\_site is followed by a comma or semi-colon so we don't get syntax errors. And that is basically how I converted a site to use cachefly for static content.

A template for a resolution for you to send to your state legislature requiring the Federal Government to reign itself back into it's Constitutional constraints and cease imposing its will on the States. Remember, the Federal Government is an agent of the States, not the other way around.

• WHEREAS, the Tenth Amendment to the Constitution of the United States reads as follows: "The powers not delegated to the United States by the Constitution, nor prohibited by it to the States, are reserved to the States respectively, or to the people"; and
• WHEREAS, the Tenth Amendment defines the total scope of federal power as being that specifically granted by the Constitution of the United States and no more; and
• WHEREAS, the scope of power defined by the Tenth Amendment means that the federal government was created by the states specifically to be an agent of the states; and
• WHEREAS, today, in 2009, the states are demonstrably treated as agents of the federal government; and
• WHEREAS, many federal laws are directly in violation of the Tenth Amendment to the Constitution of the United States; and
• WHEREAS, the Tenth Amendment assures that we, the people of the United States of America and each sovereign state in the Union of States, now have, and have always had, rights the federal government may not usurp; and
• WHEREAS, Article IV, section 4, United States Constitution, says in part, "The United States shall guarantee to every State in this Union a Republican Form of Government", and the Ninth Amendment states that "The enumeration in the Constitution, of certain rights, shall not be construed to deny or disparage others retained by the people"; and
• WHEREAS, the United States Supreme Court has ruled in New York v. United States, 112 S. Ct. 2408 (1992), that Congress may not simply commandeer the legislative and regulatory processes of the states; and
• WHEREAS, a number of proposals from previous administrations and some now pending from the present administration and from Congress may further violate the Constitution of the United States.

THEREFORE - Be it resolved by the House of Representatives of the State of <STATE>, the Senate concurring, that:

1. That the State of <STATE> hereby claims sovereignty under the Tenth Amendment to the Constitution of the United States over all powers not otherwise enumerated and granted to the federal government by the Constitution of the United States.
2. That this Resolution serves as notice and demand to the federal government, as our agent, to cease and desist, effective immediately, mandates that are beyond the scope of these constitutionally delegated powers.
3. That all compulsory federal legislation that directs states to comply under threat of civil or criminal penalties or sanctions or requires states to pass legislation or lose federal funding be prohibited or repealed.
4. That the Secretary of State of the State of <STATE> transmit copies of this resolution to the President of the United States, the President of the United States Senate, the Speaker of the United States House of Representatives, the Speaker of the House and the President of the Senate of each state's legislature and each Member of Congress from the State of <STATE>.

Replace "State of <STATE>" with your state or commonwealth and send it away. Or create this as a petition, gather signatures, then present it to your legislators. Take back your state from the Federal bureaucrats.

I am not a gun-toting fellow by any means. Typically I have waffled between whether to ban some guns or leave them all available to private citizens. I know it is a slippery slope when talking bans and I don't trust future legislators to not take advantage of a current legislator's good intentions in limiting a subsection of firearms. That said, I think it is always good to refresh your position with a well-articulated essay in support of your thoughts. And while I probably will not be toting any machine guns or bazookas, I think a handgun and some training might be time and money well spent. We'll see. Until then, here's the post that I feel articulates a good reason for the 2nd Amendment to be un-abridged by any gun laws:

The Gun is Civilization Maj. L. Caudill USMC (Ret) Human beings only have two ways to deal with one another: reason and force. If you want me to do something for you, you have a choice of either convincing me via argument, or force me to do your bidding under threat of force. Every human interaction falls into one of those two categories, without exception. Reason or force, that's it. In a truly moral and civilized society, people exclusively interact through persuasion. Force has no place as a valid method of social interaction, and the only thing that removes force from the menu is the personal firearm, as paradoxical as it may sound to some. When I carry a gun, you cannot deal with me by force. You have to use reason and try to persuade me, because I have a way to negate your threat or employment of force. The gun is the only personal weapon that puts a 100-pound woman on equal footing with a 220-pound mugger, a 75-year old retiree on equal footing with a 19-year old gang banger, and a single guy on equal footing with a carload of drunk guys with baseball bats. The gun removes the disparity in physical strength, size, or numbers between a potential attacker and a defender. There are plenty of people who consider the gun as the source of bad force equations. These are the people who think that we'd be more civilized if all guns were removed from society, because a firearm makes it easier for a [armed] mugger to do his job. That, of course, is only true if the mugger's potential victims are mostly disarmed either by choice or by legislative fiat–it has no validity when most of a mugger's potential marks are armed. People who argue for the banning of arms ask for automatic rule by the young, the strong, and the many, and that's the exact opposite of a civilized society. A mugger, even an armed one, can only make a successful living in a society where the state has granted him a force monopoly. Then there's the argument that the gun makes confrontations lethal that otherwise would only result in injury. This argument is fallacious in several ways. Without guns involved, confrontations are won by the physically superior party inflicting overwhelming injury on the loser. People who think that fists, bats, sticks, or stones don't constitute lethal force watch too much TV, where people take beatings and come out of it with a bloody lip at worst. The fact that the gun makes lethal force easier works solely in favor of the weaker defender, not the stronger attacker. If both are armed, the field is level. The gun is the only weapon that's as lethal in the hands of an octogenarian as it is in the hands of a weight lifter. It simply wouldn't work as well as a force equalizer if it wasn't both lethal and easily employable. When I carry a gun, I don't do so because I am looking for a fight, but because I'm looking to be left alone. The gun at my side means that I cannot be forced, only persuaded. I don't carry it because I'm afraid, but because it enables me to be unafraid. It doesn't limit the actions of those who would interact with me through reason, only the actions of those who would do so by force. It removes force from the equation…and that's why carrying a gun is a civilized act.

(Not mine, but can't find original source either) For Immediate Release Washington, DC - President Barack Obama and the Democrat controlled Congress is considering sweeping legislation that will provide new benefits for many Americans. The Americans With No Abilities Act. AWNAA is being hailed as a major legislative goal by advocates of the millions of Americans who lack any real skills or ambition. 'Roughly 50 percent of Americans do not possess the competence and drive necessary to carve out a meaningful role for themselves in society,' said California Senator Barbara Boxer. 'We can no longer stand by and allow People of Inability to be ridiculed and passed over. With this legislation, employers will no longer be able to grant special favors to a small group of workers, simply because they have some idea of what they are doing.' In a Capitol Hill press conference, House Majority Leader Nancy Pelosi and Senate Majority Leader Harry Reid pointed to the success of the U.S. Postal Service, which has a long-standing policy of providing opportunity without regard to performance. Approximately 74 percent of postal employees lack any job skills, making this agency the single largest U.S employer of Persons of Inability. Private-sector industries with good records of non-discrimination against the Inept include retail sales (72%), the airline industry (68%), and home improvement 'warehouse' stores (65%). At the state government level, the Department of Motor Vehicles also has an excellent record of hiring Persons of Inability (63%). Under The Americans With No Abilities Act, more than 25 million 'middle man' positions will be created, with important-sounding titles but little real responsibility, thus providing an illusory sense of purpose and performance. Mandatory non-performance-based raises and promotions will be given so as to guarantee upward mobility for even the most unremarkable employees. The legislation provides substantial tax breaks to corporations that promote a significant number of Persons of Inability into middle-management positions, and gives a tax credit to small and medium-sized businesses that agree to hire one clueless worker for every two talented hires. Finally, the AWNAA contains tough new measures to make it more difficult to discriminate against the non-abled, banning, for example, discriminatory interview questions such as, 'Do you have any skills or experience that relate to this job?' 'As a Non-abled person, I can't be expected to keep up with people who have something going for them,' said Mary Lou Gertz, who lost her position as a lug-nut twister at the GM plant in Flint, Michigan, due to her inability to remember 'rightey tightey, lefty loosey.' 'This new law should be real good for people like me,' Gertz added. With the passage of this bill, Gertz and millions of other untalented citizens will finally see a light at the end of the tunnel. Said Senator Dick Durbin (D-IL): 'As a Senator with no abilities, I believe the same privileges that elected officials enjoy ought to be extended to every American with no abilities. It is our duty as lawmakers to provide each and every American citizen, regardless of his or her inadequacy, with some sort of space to take up in this great nation and a good salary for doing so.

Sometime in the past, perhaps a month ago, I was involved in a discussion of what, if anything, the government did that was beneficial for the common man and was not an explicitly delegated power derived from the Constitution. Then today, there was a discussion on reddit concerning that very question, and I found the comments to be enlightening. The parent-est thread of the conversation can be found here. First, the challenge was made to name government services that make government a "dangerous good" rather than a "necessary evil". The list initially created included:

• Rural electrification/Tennessee Valley Authority
• DARPA (the Internet?)
• NASA
• The National Weather Service
• US Geological Survey
• Pell grants

Those are some pretty solid programs, eh? And yet…

• Rural electrification/Tennessee Valley Authority Lots of rural ares are not under TVA or other rural electrification plans. My Grandpa remembered his dad paying $700 (in 1930's money) to a nearby utility to run power to their farm. Of course the farm was just outside of Minneapolis so there was a utility to pay. For that matter my house has power from a co-op formed because a bunch of farmers decided they wanted power and made it happen.
• DARPA (the Internet?) There were other private networks. Remember fidonet? Any could have grown. Of course the government being behind DARPA with big money was a large factor, but something would have happened. Science fiction has been full of stories that assume a world (universe) wide network.
• NASA The only one here that you can make any claim at all about doing something that we wouldn't have without them today. At what cost though? If we had spent the money NASA has got over the years on genetic research we could have dragons instead. Nobody has been to the moon in my lifetime (and I'm 34), at least with dragons I could go to the circus and see them.
• The National Weather Service Most TV and radio stations already have their own weathermen. Without the NWS they might actually have to know something about weather, instead of looking good. Considering the "weathermen" in my area this would be an improvement
• US Geological Survey There are lots of private maps.
• Pell grants There are plenty of private scolarships.

Government has done some neat things. However most would have happened anyway. All come at the cost of some other neat thing.

I'm proud to announce the newest addition to the Aimonetti family. Brosef Jeffrey and Steffi have a new son, Leon Jeffrey Michael! He was born this morning, weighed 8 lbs 8 oz, and was 21 inches in length. He, Mom, and Dad are all doing wonderful! Very exciting to have the first nephew aboard. See the first pictures at my flickr page.