Hard drives and Pacific disputes

Now reading the title of this you might wonder what Hard drives and territorial disputes in the Pacific ocean could possibly have to do with one another. As you may be privy to we experienced a hard drive shortage due to some natural disasters in 2011 and the reverberations of this can still be felt in prices today to some degree. One thing to note in all of this is that a fair share of the companies with production in Thailand and other places are owned by Japanese companies. With tensions rising between Japan and China over a set of disputed islands one might wonder if a potential conflict could exacerbate the shortage and drive prices up again.

The dispute is over small islands in the East Asian sea known as Diaoyu in China and the Senkaku in Japan. Some what recently there have been semi severe incidents in which a provocation was a real risk. An example of this recently is a Chinese vessel locking radar onto a Japanese warship. The Pacific is full of such disputes especially considering the nine dotted line map China released showing the territory they see as rightfully theirs.

But what does this have to do with hard drive supplies? Well an overlooked problem is one we faced previously as a bottle neck and that is the motor. Japan's Nidec firm which produces around 80% of the motors has some portion of its manufacturing based in China. Any conflict may reduce the production capacity of this operation and thus limit the number of hard drives available. A possible conflict could create other problems as computer components are manufactured all over East and South East Asia and a conflict between China and one of those parties may bring to bear significant barriers to trade for the duration. At the end of it the cost to trade will ultimately be paid by consumers who would have to pay premiums for technology goods whose supplies are strained.

The next question is how likely is all of this. Personally I believe China and Japan will find that it is not in their best interest to pursue a conflict, and that this is a rather unlikely scenario. It is not in China's best interest to become a belligerent power as it goes against their philosophy of a peaceful rise, which they have been using the assuage the concerns of regional powers. Japan would suffer in losing market access to China and the loss in manufacturing for companies with plants based there. Ultimately it doesn't look like it would be a positive for either power, however pride and territorial disputes can make nations act rather irrationally.

Wing it and start coding

One of the better professional experiences I've had of late is attempting to learn how the Android environment works in regards to developing for it. I was tasked with a project involving the creation of an application which really had a very simple goal. But the task seemed horribly daunting, while I had taken object oriented programming, and have messed around with a few languages in the past I had not tried to develop anything for mobile. Honestly I have found the best thing to do is just jump in and start.

Seriously, just start planning the project

As with a lot of things the first step is the hardest. I spent a long time reading through random portions of the Android API documentation (http://developer.android.com/develop/index.html). Eventually though one has to actually start designing their project and then coding it. So one day I just started jotting down that the thing was supposed to do on a white board. After writing out each individual task the application needed to achieve, it was then easy to break it down into individual methods and classes. After that you now have a path or a check list of all the things you need to learn how to do in java using the Android APIs.

For example I needed to write an app to interpret XML data, store it and then display it to the user on demand, and complete the parse/download on a background thread. So breaking it down the tasks are:

- Download XML data and parse it
- Create storage space for parsed result
- Create some sort of UI to view results stored in a Database
- Start the download/parse on some sort of regular schedule

Those 4 tasks can then be broken up into individual methods and classes. For example in using a database to store information I needed to write a Database Handler to create it, define it's schema, and define all the I/O methods (more or less the CRUD stuff). One thing that is interesting to read is Oracle's beginning guide to java which also covers object oriented thinking as it's that sort of language (link: http://docs.oracle.com/javase/tutorial/java/concepts/).

Start using Google to find tutorials for everything

In my experience with this I found that more or less everything I was trying to do had been done by someone else in the past in some form, and was documented. It's actually really easy to search for and then figure out how to write classes and methods for a whole variety of tasks. For example I needed to figure out how to parse XML and found an amazing tutorial on that portion and combined with the lessons learned from a tutorial on Sqlite (embedded database).

Outside of doing stuff with Android I've also found that Code Academy is a pretty cool place to learn about coding. The interactive projects are actually rather good and certainly do an excellent job of teaching one the way a language works. It's perfect for beginners or someone trying to pick up a new language for kicks. Here's a link: www.codeacademy.com

I suppose while this article seems a bit aimless the point is to share with you that coding is fun and easy to pick up if you look in the right places. The internet is filled with pretty much everything you need from API docs, SDK docs, and tutorials. The best part is most of is completely free. So go ahead and wing it and start coding!

Linux server performance

In my daily tasks I deal with a lot of Linux servers, and from time to time decide to tweak them for performance reasons, depending on what task they are executing. A lot of the units I'm dealing with tend to be operating a postgres database and some sort of data store for a custom application that is being run (usually via tomcat). I've found that there are three easy things to play around with in order to get the most out of the system, especially if the resources on the box are fairly limited. Those things are the swappiness value, the I/O scheduler, and use of the renice command implemented with a script called via crontab.

Swappiness

The Swappiness value is what systems administrators and engineers use to instruct the linux kernel on how aggressive the system should be in storing pages of memory on disk, as opposed to in memory. Most default installations have this value at 60 which is supposed to represent a balanced number (the range is: 0-100). In my situation where I'm running a lot of database operations I've found that a higher value seems to help free up memory for use in postgres related processes, where otherwise idle system processes may have been holding on to that memory. This has been particularly effective in situations where I have application servers with just barely enough memory to get by.

You can adjust the swappiness value two ways. The first is more of a testing/temporary measure and can be done by using the following command (via the terminal):

sysctl -w vm.swappiness=(value you'd like to set it to)

You can also make this change by editing the following file: /proc/sys/vm/swappiness . One should exercise caution when editing this file though as it does require a bit of monitoring to make sure that you aren't breaking vital processes when changing memory allocation settings.

I/O Scheduler

CFQ (Completely Fair Queuing)
If my memory is still serving me well, on most Linux distributions this is the default setting. This scheduler serves as a sort of general use setting as it has decent performance on a large number of configurations and uses ranging from servers to desktops. This scheduler attempts to balance resources evenly for multiple I/O requests, and across multiple I/O devices. It's great for things like desktops or general purpose servers.

Deadline
This one is particularly interesting as it more or less takes 5 different queues and reorders tasks in order to maximize I/O performance and minimize latency. It attempts to get near real time results with this method. It also attempts to distribute resources in a manner that avoids having a process lose out entirely. This one seems to be great for things like database servers, assuming that the bottle neck in the particular case isn't CPU time.

Noop
This is a particularly lightweight scheduler and attempts to reduce CPU latency by reducing the amount of sorting occurring in the queue. It assumes that the device(s) you are using have a scheduler of their own that is optimizing the order of things.

Anticipatory
This scheduler uses a slight delay on I/O operations in order to sort them in a manner that is most efficient based on the physical location of the data on disk. This tends to work out well for slower disks, and older equipment. The delay can cause a higher level of latency as well.

In choosing your scheduler you have to consider exactly what the system is doing. In my case as I stated before I am administering application/database servers with a fair amount of load, so I've chosen the deadline scheduler. If you'd like to read into these with a bit more detail I'd check out this Redhat article (it's old but still has decent information: http://www.redhat.com/magazine/008jun05/features/schedulers/)

You can change your scheduler either on the fly by using:
echo <scheduler> > /sys/block/<disk>/queue/scheduler

Or in a more permanent manner (survives reboot) by editing the following file:
/boot/grub.conf
You'll need to add 'elevator=<scheduler> to the kernel line.

Using renice

Part of what my boxes do is serve up a web interface for users to interact with. When there are other tasks going on and the load spikes access to this interface can become quite sluggish. In my scenario I'm using tomcat as the webservices application and it launches with a 0 nice value (the normal user priority level in a range from -15-15 with lower being more important). The problem with this is that postgres also operates on the same priority and if it is loaded up with queries they are both on equal footing when fighting for CPU time. So in order to increase the quality of the user experience I've decided to set the priority for the tomcat process to -1, allowing it to take CPU time as needed when users interact with the server. I've done this using a rather crude bash script, and an entry on crontab (using crontab -e).

The script
--

#!/bin/bash
tomcatString="$(ps -eaf|grep tomcat|cut -c10-15)"
renice -1 -P $tomcatString

--

The crontab entry:

--

*/10 * * * * sh /some/path/here/reniceTomcat

--

All the above uses are the ps,grep, and cut commands to pull the process ID and then run the renice command on that ID by streaming it in. The crontab entry just calls it on a periodic basis to make sure the process stays at that priority. In the case of the above it's doing it every 10 minutes, but it can be set to just about any sort of scheduling. To read more on how to use cron scheduling check out this article: http://www.debian-administration.org/articles/56.

Analytics in Columbus?

I read something rather interesting in the paper this morning. It would seem that IBM is putting a new analytics center right in my backyard here in Columbus Ohio. This is big news for the city as it's supposed to bring in around 500 new tech jobs as well add credibility to the region as a tech center. Data/Business Analytics is a fascinating field, and is certainly worth a gander as it represents something significant for the future of the technology sector.

You see right now all the talk is about 'big data' and how it is stored, where it is served from, how its collected. But the lingering question that a lot of companies are now answering is 'what do you do with it once it's there?'. Companies such as IBM are taking this data boiling it down and using it to formulate strategies, see patterns of behavior that might be uncouth, and where consumer interest is going. This trend has caused a new type of IT job to exist that is an interesting mix of both technological, and business savvy.

What all of this means for those of us here in the Midwest is that its a step toward breaking the assumption that all of the IT talent is on either the east or west coast of the US. So all in all this should be a positive sign for the economy here, plus I must add it will be interesting to see the sort of talent that Ohio State is able to churn out for this field. To that end the Fisher College is opening up a new Graduate program for it, and the college itself is looking into something in the undergrad arena.

There should be some interesting times ahead for the Tech sector in Columbus.

Deficit Hawk: A cool federal budget app

So due to my being a bit of a public policy nerd on top of my enjoyment of technology I started playing around with an app on the Google Play Store called 'Deficit Hawk'. A friend of mine had suggested it, and I must say its a neat little app. It takes CBO projections, and possible choices that cover new revenues as well as cuts and allows you to attempt to set a budget plan. It's incredibly easy to use, and gives you a nice graph so you can see how you are doing.

The plan I created when messing around with it is below:

----- NEW SPENDING CUTS -----

$-88.0 billion over ten years - Add a Public Plan to the Health Insurance Exchanges

$-88.5 billion over ten years - Apply the Social Security Benefit Formula to Individual Years of Earnings

$-112.0 billion over ten years - Base Social Security Cost-of-Living Adjustments on an Alternative Measure of Inflation

$-2.0 billion over ten years - Charge transactions fees to fund the Commodity Futures Trading Commission

$-4.8 billion over ten years - Drop Wealthier Communities from the Community Development Block Grant Program

$-20.8 billion over ten years - Increase Fees for Aviation Security

$-26.5 billion over ten years - Increase Guarantee Fees Charged by Fannie Mae and Freddie Mac

$-241.2 billion over ten years - Increase the Basic Premium for Medicare Part B to 35 Percent of the Program's Costs

$-85.6 billion over ten years - Limit Highway Funding to Expected Highway Revenues

$-62.4 billion over ten years - Limit Medical Malpractice Torts

$-84.6 billion over ten years - Link Initial Social Security Benefits to Average Prices Instead of Average Earnings|Implement progressive price indexing

$-124.8 billion over ten years - Raise the Age of Eligibility for Medicare to 67

$-119.9 billion over ten years - Raise the Full Retirement Age in Social Security

$-642.0 billion over ten years - Reduce Growth in Appropriations for Agencies Other Than the Department of Defense|Freeze Funding at 2011 Level

$-610.7 billion over ten years - Reduce the Growth in Appropriations for the Department of Defense|Freeze Funding at 2011 Level

$-112.0 billion over ten years - Require Manufacturers to Pay a Minimum Rebate on Drugs Covered Under Medicare Part D for Low-Income Beneficiaries

$-3.6 billion over ten years - Transfer the Tennessee Valley Authority's Electric Utility Functions and Associated Assets and Liabilities

----- NEW REVENUE -----

$309.5 billion over ten years - Accelerate and Modify the Excise Tax on High-Cost Health Care Coverage

$96.1 billion over ten years - Expand Social Security Coverage to Include Newly Hired State and Local Government Employees

$241.4 billion over ten years - Extend the Period for Depreciating the Cost of Certain Investments

$70.9 billion over ten years - Impose a Fee on Large Financial Institutions

$456.8 billion over ten years - Increase the Maximum Taxable Earnings for the Social Security Payroll Tax

$1.2 trillion over ten years - Limit the Tax Benefit of Itemized Deductions to 15 Percent

$48.7 billion over ten years - Raise Tax Rates on Capital Gains

--------

In any case if you have an interest in public policy, and you enjoy playing around with neat apps on your phone or tablet I suggest giving this a go.

What is data protection really?

Data protection is a vague term that I've seen being thrown about. I've observed it being used in reference to data backup software, security software, network security devices, and well really to all sorts of software and hardware platforms and devices. I've come to the conclusion that data protection is some bit of all of these things and ultimately is about three factors: protection from loss, protection against leak, and the ability to ensure the data can be trusted.

Data loss is perhaps the most fiscally costly, and best known portion of data protection. It is here that you have your nightmare scenarios about all the customer data being gone, and your IT staff is rung up in the middle of the night to rush around to save what they can. These days most are conducting some form of data backup to cover themselves here, and the wiser of us are doing so to off site locations. Obviously this one is something that every organization should have covered at the minimum, though if you aren't sure this is taken care of go ahead and take a look at this post that kicks off a series on backup solutions, and this one which is a webinar about backup solutions.

Data leak is perhaps equally dangerous, though not as often thought about. This area of data protection involves the act of a malicious party gaining access to your sensitive information for some sort of nefarious purpose. A leak can harm not only those of whom the data belongs to or references, but also the reputation of the organization that has been breached.

The risks of leak are present in three stages, transmission, storage, and actions by the internal users themselves. It is important then that all transmitted communications have some form of encryption enabled, be they emails, backups, or other web transactions between customers or internal users. Sending anything across in plain text is just asking for a breach. These days most sites and services provide this, and just about every email service be it hosted or self hosted is capable of some level of protection here. Encryption of stored data is also rather important, and is increasingly so for those with laptops on the road. I can say that I've had my personal information exposed to the world on two occasions due to a laptop being stolen. It's a frustrating circumstance, and can cause all sorts of havoc for a business. It's important to do some for of encryption on your laptops, and its easy with the free solutions out there (like truecrypt).

Trust is important as well. When you are backing something up, or interacting with a web service how can you be assured that you are accessing data that is without malicious content. There are solutions that do this in some sort of piecemeal format be it scanning your computer to make sure it's up to date, or doing general scans to ensure OS integrity, but I'm not aware of a true comprehensive point to point solution. When interacting with questionable web services I might suggest running some sort of sandbox utility, such as that offered by avast, or sandboxie.

Why your SMB needs a private cloud

For small to medium size enterprises (SMB) the private cloud is the next natural step for their IT infrastructure. Their employees need access to data no matter where they are, and the employer needs them to be always on, and always connected. The organization is going to want to accomplish this goal in a manner that is cost effective, secure, and able to be owned.

Employees need their data. Almost every job these days from sales to engineering requires access to either some form of CRM (customer relations management) interface, ticketing system, knowledge base, or some other database oriented solution. Setting up some form of private cloud environment that is accessible from anywhere is the key to giving them what they need. It's becoming easier and easier to set up an infrastructure these days as well. You really just need some blade servers and some sort of virtualization platform tossed on top in order to deploy virtual machines to serve your purposes. Ultimately your deployment becomes less about hardware and more about services and software.

Your infrastructure cost should go down over time as well. While you wont be able to layoff your IT team, they will be able to automate more tasks, centralize more of the infrastructure, and spend time on things that develop the business instead of fighting fires. If you set up more centralized infrastructure with a proper disaster recovery and business continuity plan utilizing backup essentials you can create a resilient and accessible set of services for your employees and customers.

Ownership is also an issue of great import. One of the next big things in IT is going to be the 'how' in determining what filters to place on all this data that has been collected on users. The answer to that question is going to bring up a lot of privacy concerns, as well as the issue of user rights v. owner rights. Through the use of a private cloud you can avoid some of those pitfalls that you would run straight into by going to a public provider. The key portion of this is that in a private cloud scenario your data is housed in something you physically own. If you need to pull something, or migrate the data away you can and without being impeded by the governance of another organization outside of your own.

In owning the infrastructure yourself you can also lessen the risk of data leak. You have the opportunity to set your own strict testing and security standards. In the hands of another company your are subject to what ever policies they have dreamt up, be they for better or worse. This sort of thing is especially important if you are storing proprietary data, or personally identifiable customer/employee information.

The move to the private cloud is a natural step forward. We've witnessed over the past twenty years the empowerment of the personal desktop, which lead to the beginnings of a collaborative office environment. Now we're moving to each organization having it's own private cloud of computing power, giving them further capabilities and control. Your organization can move from a collaborative office environment to being that of a collaborative organization. No matter where your employees are, they can communicate and in a manner that is under your control.