SUSE Studio

I might be late in the game on discovering Suse Studio, but I must admit that I felt compelled to talk about it a bit today. Now in the past I really haven't used the Suse distribution of Linux so much, as I had already fallen into the use of others such as Fedora, CentOS, Debian, Ubuntu, Mint, etc and just hadn't made my way down the list to them yet. But then they released this web interface with the purpose of making it easy to build a customized Linux image or appliance. My home lab is now going to change a bit.

The reason I say my home lab is going to change, is because of how simple it really is Suse Studio to make a Linux VM has the exact services I want, and without me having to deal with the hassle of setting them up manually. To show you just how much of a time saver this is I'm going to go ahead and walk you through the creation of a VMware appliance.

For the purposes of this we're going to set up a fairly simple web server. To start we are presented with the choice of a base template. I'm going to go ahead and choose the server addition of Suse Enterprise 11 with a 32bit architecture.

Next we'll go into the software tab and take a look at that menu. To start off with there are a little over 300 packages selected for install. For my case I've decided to add apache2, the PHP mod for apache and postgresql-server as additional packages. This would ostensibly give me a good basis for a webserver with some decent functionality.

 

Next one can begin to set their custom configuration for the VM. Here I'll create a user account called 'tux' to use. You can also do things like customize the network settings, but for the purposes of this example I'll leave it alone at DHCP. Two things that are really cool here though are that you can use a SQL dump to set up the schema for your database in the image, saving the time of having to write create table queries. And you can also customize the VM appliance hardware settings so that you don't have to mess with it when you import the VM files into your environment.

Lastly before going into the build phase you can also upload any files you would like to have on the machine to start out with. This could be useful in a variety of scenarios, such as ours if we were migrating our web site from one server to another. And once you've finished that up we move into the build phase. The build phase gives a whole lot of options as far as format for your new appliance. You can get it as an ISO, VM files (various vendors) or even as an EC2 image. For this I went ahead and had it generate a Vmware VM appliance. You can also select more than one format which is cool.

The build for me took around 8 minutes which really is rather fast. The whole process to do this took less than 15 minutes in order to get through the menus, and we now have a viable image for a web server. They have also provided the ability to test drive the appliance for up to an hour, and the ability to export the image configuration for local building (they delete the build after 7 days forcing you to have to click build again and wait the few minutes). In any event this is a neat web based tool that I'll certainly be playing with quite a bit.

Also, keep an eye out, as I am being tasked to write some articles in other interest areas such as international relations in the near future. There should be some good content both on technology, and other areas in the days to come.

Ryan Koch
3X Systems
ryan.koch@3x.com

Failover clustering (My simple test environment)

A post two days in a row? This is some sort of record. In any case today we're going to discuss the small and not best practices friendly test environment I set up here at 3X. The reason the cluster exists is that we had to adjust the product to suit the needs of a customer who has a SQL cluster, and as such we needed a test environment with which to pound on incessantly to make sure it worked. I'll very quickly detail what went into it, and then describe how it is being backed up by 3X Appliance.

Here is a list of the things needed in order to set it up:

1. 2 VMs running Windows Server 2008 R2 Enterprise or Datacenter (Mine are datacenter)
2. 1 VM running a virtual NAS device to act as shared storage (FreeNAS for me)
3. Windows domain
4. Backup appliance

So the list of parts is pretty simple. In my instance I have a VM set up on one of our hyper-v test hosts, and another set up on my test ESXi host (I do enjoy the mixed environment). The first step I had to complete after getting the operating system set up for both VMs and getting them on the domain properly was to install the failover cluster feature. It's a straight forward process just like any other Windows Server feature install and I merely followed the wizard as it lead the way.

Now before you set up the actual cluster you have to make sure there is some sort of shared storage setup. Now Windows Server requires Persistent Reservations to be enabled on this storage so you can't use OpenFiler in order to simulate the NAS. After some digging around I did find out that FreeNAS supports said feature.You can follow the instructions here in order to set up an appliance to run the FreeNAS software (unless of course you are lucky enough to have a NAS already). And these instructions are helpful for setting up the iSCSI target you'll need to use as the shared storage. After FreeNAS is set up right you'll just need to use the Microsoft iSCSI initiator to get the LUN exposed to your VMs. You must do this on all VMs you are adding to the cluster in order for it to work properly.

Next you'll need to go into your snap in console and to the Clustering management portion in order to actually create the cluster. Again this is merely a matter of going through a few wizard windows, but as long as you have all the prerequisites set up properly you shouldn't have any trouble.Once complete you'll have a lovely two or more node cluster for your testing/enjoyment. Pay particular attention to the validation step for the wizard as it's very good at telling you all the things you forgot to do, or did wrong. One thing I should note here is that you should never use a two node cluster for production. If one of the nodes fails the whole thing goes down and you lose quorum. At minimum you should have 3 nodes for a production environment.

Now on to backing it up. Since I work at 3X and have access to their appliances I use one of them to backup the evironment. I have a SQL level and a file level backup taking place for this cluster. Now there is something critical to keep in mind if  you are using an agent based backup like ours to backup a cluster and that is the ownership of the LUN(s) you are using for shared storage. In my example I have sqlcluster01 and sqlcluster02.

As you can see from the screenshot my disk is owned by sqlcluster02. So the only way I'm going to get a file level back up of this volume is if I have the agent installed on that VM. Once this is achieved you'll see the drive show up as any other in our wizard as below:

 So file level backup is pretty straight forward, but what about SQL? Well for 3X as of version 3.1.25 you can backup SQL clusters and you do so just like any other SQL instance. You just install the agent on the production node and set up a SQL job as per the instructions you can find here under Administrative Guide > Creating Backups.

That's the summary of things. I am of course always available for any feed back or questions which you can leave either in the comments or you can email me directly via the address in my signature block below.

Ryan Koch
3X Systems
ryan.koch@3x.com

Thoughts on the HP touchpad

Now I was not one of the lucky few that had caught the Touchpad when it went for 99 dollars a while back. However, I did notice fairly recently that Newegg had it refurbished for 230 (it's now a bit more expensive) (link), which really isn't bad. I ended up purchasing this tablet, mostly as a toy that I could fool around with as I wanted to installed ICS on it. As it turns out Cyanogen Mod 9 is out for it and in alpha (now officially). So, I figured I'd share a couple tips with anyone else brave enough to make this purchase and to share some opinion of its performance.

Specs:


CPU Type
    Qualcomm Snapdragon dual-core

CPU Speed
    APQ8060(1.2GHz)

Screen Size
    9.7"

Resolution
    1024 x 768

GPU/VPU
    Adreno 220

Hard Drive

HDD
    32GB Storage

Memory
    1GB

WLAN
    Dual-band Wi-Fi 802.11a/b/g/n

Bluetooth
    Bluetooth 2.1

Battery
    6300mAh (typical) lithium-polymer battery

Physical spec

Dimensions
    9.45" x 7.48" x 0.54"

Weight
    1.6 lbs


Out of the box it had WebOS installed. Having not used a device with this operating system I was actually impressed by the UI and in particular how it handled switching between running applications with its card system. The problem with it, as has been addressed by many bloggers is the complete lack of applications for it. This is a large reason why I decided to go ahead and install android next it in a dual boot fashion, as WebOS would stand as an OS I could use if I blow Android up playing with it.

The installation of Cyanogen 9 is actually pretty easy. You can follow the instructions found here. After the installation I was surprised to find that despite being in Alpha the OS ran rather well. The camera is still inoperative and there are some buggy situations you'll run into, but overall I found the tablet tolerable to use in this scenario as a normal driver. I did however run into a couple problems that required some digging around the internet to solve.

The biggest problem turned out to be charging at first. I ran into a situation where the tablet would not charge unless it was booted into WebOS. The behavior was obviously software related and so the search for a solution began. I ended up finding out that the storage options needed to be changed in order for the current build of CM 9 to charge inside android. So I went to the Settings application, then to the button on the top right for more options and hit USB computer connection. There one needs to set the device to MTP mode. The other benefit to this change is that it should let you mount the device in windows as well for getting at the internal SD card storage. The moment I made this change the behavior improved and I can now charge the device (hooray!).

Outside of that charging issue I've had a true pleasure in playing with the device. The interface is fast, and ICS is a huge improvement over earlier versions of Android. The tablet has been excellent in use for email, Teamviewer (IT type stuff), Social networks, Netflix, and really all any other tasks that you'd also complete with a smart phone. The device does feel a bit heavy, and the sleeve it comes with from Newegg is mediocre. I definitely suggest getting the HP made case for it, and if you feel like springing for it the touchstone charger as both enhance the ownership experience quite a bit. In any event if you are looking for a fairly cheap tablet with decent specs its not a bad choice if you're willing to get your hands dirty.

Ryan Koch
3X Systems
ryan.koch@3x.com

Exchange Maintenance

I apologize for having disappeared for a short bit. I had some military training that pulled my attention from this blog for a short time. However, jumping back into the swing I'd like to make a short note about Exchange maintenance policy. For those of you that are using a backup solution utilizing VSS to complete file backups of the information store, this is of particular importance.

I have found that a lot of our clients have the maintenance cycle running as a daily occurrence. The logic of this move is that having the defrag occur that often is going to make the database more efficient and thus keep performance up for end users. The trouble is that if you attempt to use any backup solution that scans at the block level you are going to see a lot of change data. The consequence of this is that your backups will take longer than they should because you're sending over all these changed blocks that really aren't new data, but are flagged as such due to the shuffle. More often may not always be better.

A good real world example of this is a customer I had recently. They had a fairly small information store of around 50-60GB and were seeing backup jobs (using the 3X backup appliance) taking as long as 5-6 hours. Upon investigation it was found that each day the Exchange client was transmitting upwards of 4 GB of change data. I worked with the customer to reduce the scheduling down to twice a week over night (not conflicting with the backup window). The results of the change were staggering. After the first day we saw a 20% improvement, and after a week the job was down to 2 hours (60%).

Overall I suggest attempting to run maintenance less often if possible. Smaller deltas should speed up your backup solution, and cause less saturation on your infrastructure from the operation.

Ryan Koch
3X Systems
ryan.koch@3x.com

Highly volatile postgresql

I've had a few clients that have had some issues with postgresql and performance problems taking place over time due to their databases being highly transactional. Mind you these customers are really dealing with a single database that has grown in size, as opposed to several on a single server. It had gotten to the point where my colleague Rich Keggans (from 3X Systems also) had been scheduling a monthly cluster to be run due to how bad the performance would become.

So in response we tried a little experiment involving tweaking the auto-vacuum settings and creating an automated task. I felt the need to share it with you, as we've seen some rather promising results so far as performance degradation seems to have ceased in two instances so far.

For the auto-vacuum we really just made the settings a bit more aggressive. In the case we tested this the customer's database had a very high level of transactions taking place. The chance for bloat to build up here is pretty high so we made the settings reflect the following in order to fight it:

autovacuum = on                         # enable autovacuum subprocess?
                                        # 'on' requires stats_start_collector
                                        # and stats_row_level to also be on
autovacuum_naptime = 1min               # time between autovacuum runs
autovacuum_vacuum_threshold = 250       # min # of tuple updates before
                                        # vacuum
autovacuum_analyze_threshold = 150      # min # of tuple updates before
                                        # analyze
autovacuum_vacuum_scale_factor = 0.2    # fraction of rel size before
                                        # vacuum
autovacuum_analyze_scale_factor = 0.1   # fraction of rel size before
                                        # analyze
autovacuum_freeze_max_age = 200000000   # maximum XID age before forced vacuum
                                        # (change requires restart)
autovacuum_vacuum_cost_delay = -1       # default vacuum cost delay for
                                        # autovacuum, -1 means use
                                        # vacuum_cost_delay
autovacuum_vacuum_cost_limit = -1       # default vacuum cost limit for
                                        # autovacuum, -1 means use
                                        # vacuum_cost_limit

So as you can see we changed a couple values by a small amount in order to make it more aggressive. Paired with this I wrote a quick one line script that would run Analyze. The analyze should update the statistics for the table which makes postgresql's rather effective query planner work at its best. The following one liner is the script, which I had put into /sbin/:

 psql -c "analyze" -q -S -d boxicom -U boxicom > /usr/vault/log/pganalyzetask.log

This was paired with an entry in the crontab using crontab -e:

*   4,20   *   *   * /sbin/pganalyzetask

In this case I picked 4am and 8pm due to the fact that the customer had constant activity during normal business hours and again during a certain period at night. This ideally is going to update the stats after those periods allowing his scripts to perform well on each subsequent time period. 

Further tweaking is going to be required to get these servers running just right, but this is a start anyway. I'll likely be diving back into the postgresql.conf file to see what other changes we can make (based on the hardware available of course). Hopefully we'll continue to see performance improvements on these postgresql databases.

Ryan Koch
3X Systems
ryan.koch@3x.com