<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>BVLog &#187; sysadmin</title>
	<atom:link href="http://bvlog.vosswerx.com/category/sysadmin/feed/" rel="self" type="application/rss+xml" />
	<link>http://bvlog.vosswerx.com</link>
	<description>Bryan Voss' mental synchronization point</description>
	<lastBuildDate>Fri, 30 Jul 2010 04:01:13 +0000</lastBuildDate>
	<generator>http://wordpress.org/?v=2.8.4</generator>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>Big Data</title>
		<link>http://bvlog.vosswerx.com/2010/06/29/big-data/</link>
		<comments>http://bvlog.vosswerx.com/2010/06/29/big-data/#comments</comments>
		<pubDate>Tue, 29 Jun 2010 22:43:35 +0000</pubDate>
		<dc:creator>bvoss</dc:creator>
				<category><![CDATA[datacenter]]></category>
		<category><![CDATA[hardware]]></category>
		<category><![CDATA[san]]></category>
		<category><![CDATA[sysadmin]]></category>

		<guid isPermaLink="false">http://bvlog.vosswerx.com/?p=408</guid>
		<description><![CDATA[I just added another 15 terabytes of disk to one of the SANs at work that I manage. Woohoo! Always fun dealing with lots of storage. Now off to provision some new datastores for VMWare ESX.
]]></description>
			<content:encoded><![CDATA[<p>I just added another 15 terabytes of disk to one of the SANs at work that I manage. Woohoo! Always fun dealing with lots of storage. Now off to provision some new datastores for VMWare ESX.</p>
]]></content:encoded>
			<wfw:commentRss>http://bvlog.vosswerx.com/2010/06/29/big-data/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Creating/deleting Windows services</title>
		<link>http://bvlog.vosswerx.com/2010/06/28/creatingdeleting-windows-services/</link>
		<comments>http://bvlog.vosswerx.com/2010/06/28/creatingdeleting-windows-services/#comments</comments>
		<pubDate>Mon, 28 Jun 2010 21:39:43 +0000</pubDate>
		<dc:creator>bvoss</dc:creator>
				<category><![CDATA[sysadmin]]></category>
		<category><![CDATA[windows]]></category>

		<guid isPermaLink="false">http://bvlog.vosswerx.com/?p=400</guid>
		<description><![CDATA[I always have to look the procedure up when I need to create/delete a Windows service, so here it is for posterity.
Create service (note the space after binPath=):
sc create "[service name]" binPath= "[path to executable]"
Delete service:
sc delete "[service name]"
&#60;rant&#62;
And as long as we're on the subject, I really wish our "enterprise" software vendors would stop [...]]]></description>
			<content:encoded><![CDATA[<p>I always have to look the procedure up when I need to create/delete a Windows service, so here it is for posterity.</p>
<p>Create service (note the space after binPath=):<br />
<code>sc create "[service name]" binPath= "[path to executable]"</code></p>
<p>Delete service:<br />
<code>sc delete "[service name]"</code></p>
<p>&lt;rant&gt;<br />
And as long as we're on the subject, I really wish our "enterprise" software vendors would stop setting things up to run as Windows console apps when all they do is display a logfile in a window. If the app has the ability to run as a service, run it as a service. I can tail a logfile to debug issues, even if your support people don't know how to do that.</p>
<p>Running apps on the console tends to cause problems if people login via RDP. I can either:</p>
<ol>
<li>Explain to people that they have to login via RDP using a particular account with an obscure password in order to restart an app, making sure to include a "/console" after mstsc on the commandline.</li>
<li>Tell them to login using their own account, double-click the Services icon on the desktop, right-click the app on the list and click Start.</li>
</ol>
<p>Hmm. I think I prefer option 2.<br />
&lt;/rant&gt;</p>
]]></content:encoded>
			<wfw:commentRss>http://bvlog.vosswerx.com/2010/06/28/creatingdeleting-windows-services/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Clariion hosts showing as unmanaged</title>
		<link>http://bvlog.vosswerx.com/2010/06/15/clariion-hosts-showing-as-unmanaged/</link>
		<comments>http://bvlog.vosswerx.com/2010/06/15/clariion-hosts-showing-as-unmanaged/#comments</comments>
		<pubDate>Tue, 15 Jun 2010 13:32:59 +0000</pubDate>
		<dc:creator>bvoss</dc:creator>
				<category><![CDATA[datacenter]]></category>
		<category><![CDATA[san]]></category>
		<category><![CDATA[sysadmin]]></category>
		<category><![CDATA[windows]]></category>

		<guid isPermaLink="false">http://bvlog.vosswerx.com/?p=391</guid>
		<description><![CDATA[I have had several Windows servers connected to an EMC Clariion SAN via both Fibre Channel &#038; iSCSI show up as unmanaged, even though they all have Navisphere Agent installed and running. After some investigation, I found that all of the hosts have multiple NICs, either for cluster heartbeat purposes or for iSCSI connectivity. In [...]]]></description>
			<content:encoded><![CDATA[<p>I have had several Windows servers connected to an EMC Clariion SAN via both Fibre Channel &#038; iSCSI show up as unmanaged, even though they all have Navisphere Agent installed and running. After some investigation, I found that all of the hosts have multiple NICs, either for cluster heartbeat purposes or for iSCSI connectivity. In Navisphere, right-clicking the host and choosing "Update Now" gave an error which included the IP of one of the private interfaces. In other words, the agent is binding to the wrong adapter.</p>
<p>Solution:</p>
<ol>
<li>Create a file named "agentid.txt" under the Navisphere Agent directory.</li>
<li>The first line of the file should contain the server's fully-qualified hostname.</li>
<li>The second line should contain the IP address that Navisphere should use to contact the server. This determines which adapter will be used.</li>
<li>Stop/start Navisphere Agent service. Do not restart the service, as that doesn't seem to work.</li>
</ol>
<p>In Navisphere, right-click on the host and click "Update Now". It should show up as managed.</p>
]]></content:encoded>
			<wfw:commentRss>http://bvlog.vosswerx.com/2010/06/15/clariion-hosts-showing-as-unmanaged/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>FTP only shell for ProFTPd</title>
		<link>http://bvlog.vosswerx.com/2010/06/10/ftp-only-shell-for-proftpd/</link>
		<comments>http://bvlog.vosswerx.com/2010/06/10/ftp-only-shell-for-proftpd/#comments</comments>
		<pubDate>Thu, 10 Jun 2010 16:57:20 +0000</pubDate>
		<dc:creator>bvoss</dc:creator>
				<category><![CDATA[linux]]></category>
		<category><![CDATA[sysadmin]]></category>

		<guid isPermaLink="false">http://bvlog.vosswerx.com/?p=376</guid>
		<description><![CDATA[I recently set up ProFTPd on an externally-accessible Ubuntu Linux server to allow remote users to download software updates. In order to lock the user account down somewhat, I set the account's shell to /bin/false in /etc/passwd . I thought that would allow FTP access without allowing the account to login via ssh. Unfortunately, that [...]]]></description>
			<content:encoded><![CDATA[<p>I recently set up ProFTPd on an externally-accessible Ubuntu Linux server to allow remote users to download software updates. In order to lock the user account down somewhat, I set the account's shell to /bin/false in /etc/passwd . I thought that would allow FTP access without allowing the account to login via ssh. Unfortunately, that did not work. The account was denied access because ProFTPd was checking to make sure the shell exists in /etc/shells .</p>
<p>I did a little googling and found a <a href="http://www.experts-exchange.com/OS/Linux/Administration/Q_20788215.html">solution</a> on the somewhat-horrendous-but-ubiquitous Experts Exchange site. With a little tweaking, it works great.</p>
<ol>
<li>Create a file: /bin/ftponly<br/><br />
<code>#!/bin/bash<br />
echo "This account is only allowed FTP access."</code><br/><br />
<br/>
</li>
<li>Add /bin/ftponly to /etc/shells file</li>
<li>Change shell for user account either by using chsh or editing /etc/passwd directly</li>
</ol>
<p>Once this is done, ProFTPd considers ftponly a valid shell and allows FTP access. As a side bonus, attempting to login via ssh or other means displays a useful message rather than just dropping the connection.</p>
<p>Now, on to finding a solution for the continuous brute force login attempts that are filling up the ProFTPd logfiles...</p>
]]></content:encoded>
			<wfw:commentRss>http://bvlog.vosswerx.com/2010/06/10/ftp-only-shell-for-proftpd/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Linux force password change</title>
		<link>http://bvlog.vosswerx.com/2009/09/14/linux-force-password-change/</link>
		<comments>http://bvlog.vosswerx.com/2009/09/14/linux-force-password-change/#comments</comments>
		<pubDate>Mon, 14 Sep 2009 19:27:29 +0000</pubDate>
		<dc:creator>bvoss</dc:creator>
				<category><![CDATA[linux]]></category>
		<category><![CDATA[sysadmin]]></category>

		<guid isPermaLink="false">http://bvlog.vosswerx.com/?p=297</guid>
		<description><![CDATA[I needed to have a user change their password on their next login, so I had to look up how to do that. Since I had to do a little more searching than usual to find an answer, I'm posting it here for posterity.
First, change the user's password to a temporary:
passwd [username]
Next, reset the password [...]]]></description>
			<content:encoded><![CDATA[<p>I needed to have a user change their password on their next login, so I had to look up how to do that. Since I had to do a little more searching than usual to find an answer, I'm posting it here for posterity.</p>
<p>First, change the user's password to a temporary:<br />
<code>passwd [username]</code></p>
<p>Next, reset the password expiration to 90 days (-M) and set the last change date to 0 (-d) to force a change:<br />
<code>chage -M 90 -d 0 [username]</code></p>
<p>Finally, verify the info:<br />
<code>chage -l [username]</code></p>
]]></content:encoded>
			<wfw:commentRss>http://bvlog.vosswerx.com/2009/09/14/linux-force-password-change/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Windows: svchost.exe taking up a lot of memory</title>
		<link>http://bvlog.vosswerx.com/2009/07/06/windows-svchostexe-taking-up-a-lot-of-memory/</link>
		<comments>http://bvlog.vosswerx.com/2009/07/06/windows-svchostexe-taking-up-a-lot-of-memory/#comments</comments>
		<pubDate>Mon, 06 Jul 2009 15:38:14 +0000</pubDate>
		<dc:creator>bvoss</dc:creator>
				<category><![CDATA[sysadmin]]></category>
		<category><![CDATA[windows]]></category>

		<guid isPermaLink="false">http://bvlog.vosswerx.com/?p=282</guid>
		<description><![CDATA[One of my coworkers has been keeping an eye on a couple of Windows Server 2003 boxes at work due to some problems we have had with them. He sent me a screenshot of Task Manager showing a svchost.exe instance using more than 1GB of memory. Unfortunately, by the time I checked the server, the [...]]]></description>
			<content:encoded><![CDATA[<p>One of my coworkers has been keeping an eye on a couple of Windows Server 2003 boxes at work due to some problems we have had with them. He sent me a screenshot of Task Manager showing a svchost.exe instance using more than 1GB of memory. Unfortunately, by the time I checked the server, the process was gone.</p>
<p>Since svchost.exe can host many things, I did a little research and sent the following command to him to help us identify what's consuming so much memory:<br />
<code>tasklist /svc /fo list /fi "imagename eq svchost.exe" /fi "memusage gt 1000000"</code></p>
<p>That will show any svchost.exe processes that are consuming more than 1GB of memory. It will also list all the hosted DLLs so we can track down the culprit. Of course, all the same info is available in <a href="http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx">Process Explorer</a>, but we don't have that installed on the box in question (although I am thinking of suggesting that we make that a standard part of our server loadout).</p>
<p>I may also encapsulate the tasklist command above into a Nagios check and run it against all our Windows boxen. That's what I like about having a flexible monitoring system like Nagios in place. As we find new things to check, we can just add them and the whole thing is automated from that point on.</p>
]]></content:encoded>
			<wfw:commentRss>http://bvlog.vosswerx.com/2009/07/06/windows-svchostexe-taking-up-a-lot-of-memory/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Linux process elapsed time</title>
		<link>http://bvlog.vosswerx.com/2009/05/22/linux-process-elapsed-time/</link>
		<comments>http://bvlog.vosswerx.com/2009/05/22/linux-process-elapsed-time/#comments</comments>
		<pubDate>Fri, 22 May 2009 20:43:46 +0000</pubDate>
		<dc:creator>bvoss</dc:creator>
				<category><![CDATA[linux]]></category>
		<category><![CDATA[sysadmin]]></category>

		<guid isPermaLink="false">http://bvlog.vosswerx.com/?p=269</guid>
		<description><![CDATA[I'm working on a script to dump a daily audit log from one of our clinical systems. The script invokes Oracle's sqlplus and runs a query provided by the vendor to show all user activity in the past 24 hours. I started the query running with the expectation that it would take at most a [...]]]></description>
			<content:encoded><![CDATA[<p>I'm working on a script to dump a daily audit log from one of our clinical systems. The script invokes Oracle's sqlplus and runs a query provided by the vendor to show all user activity in the past 24 hours. I started the query running with the expectation that it would take at most a few minutes to run. That was before lunch. It's now late afternoon and the script is still running.</p>
<p>The output file is slowly growing, so it's still working. I attached an strace to the process and see it mostly waiting in a read state, so I assume that we just need to optimize the indexes on the database to make it run faster. That's mostly outside my responsibility, but I did want to check to see how long the process had been running in order to make a preliminary report to my coworkers. Digging around in the /proc directory for the process didn't immediately show me what I wanted to know, so I turned to the "ps" command. I had to do some reading, but ended up with the following:<br />
<code>ps -o etime,stime,time,cmd -C sqlplus</code></p>
<p>This gave me what I was looking for: elapsed time, system time, wait time, and the command for all sqlplus processes.</p>
<p>Total elapsed time so far: 5 hours. Wow. That's quite a while for a single query.</p>
]]></content:encoded>
			<wfw:commentRss>http://bvlog.vosswerx.com/2009/05/22/linux-process-elapsed-time/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>1.) Is it turned on?</title>
		<link>http://bvlog.vosswerx.com/2009/03/09/1-is-it-turned-on/</link>
		<comments>http://bvlog.vosswerx.com/2009/03/09/1-is-it-turned-on/#comments</comments>
		<pubDate>Mon, 09 Mar 2009 06:23:44 +0000</pubDate>
		<dc:creator>bvoss</dc:creator>
				<category><![CDATA[datacenter]]></category>
		<category><![CDATA[hardware]]></category>
		<category><![CDATA[sysadmin]]></category>

		<guid isPermaLink="false">http://bvlog.vosswerx.com/?p=244</guid>
		<description><![CDATA[I was recently contacted by EMC Support saying they had not received a health report from our new secondary Centera cluster in a while. They had tried dialing into the cluster via modem, but were not getting a response. They asked me to reset the modem on the cluster to ensure that it was working [...]]]></description>
			<content:encoded><![CDATA[<p>I was recently contacted by EMC Support saying they had not received a health report from our new secondary Centera cluster in a while. They had tried dialing into the cluster via modem, but were not getting a response. They asked me to reset the modem on the cluster to ensure that it was working correctly so they could dial in and check things out.</p>
<p>As soon as I hung up the phone, I brought up my Centera Viewer client and tried to login to the cluster. No response. No ping response either. As I walked down the hall to the datacenter, I was reviewing network connectivity for the cluster in my mind. If a system isn't working correctly, blame it on the network, right?</p>
<p>Once in the datacenter, I opened the back of the rack and found the modem dead. No lights at all. After checking cables, it occurred to me that I wasn't feeling any breeze from the fans in all the nodes. A quick glance told me that there were no lights on the back of the cluster. I walked around to the front and found no lights there either.</p>
<p>As possible causes for a complete power failure to the rack began whizzing through my head, one tidbit floated to the surface: About two weeks before, we had been coordinating with the Maintenance department on moving our datacenter power feeds to a new powerhouse the hospital recently built. We have big APC UPSes that will power the datacenter for a few minutes until generators kick in. Since Maintenance wasn't sure how long it would take to reroute power through the new powerhouse and generators were out of the question, we had to prepare for the worst and assume the UPSes would drain and shut down before power was restored. One of the steps we took was powering down all non-critical systems. Since the new Centera was a replication target and replication was not in full swing yet, I decided to power it down for the move.</p>
<p>Of course, I'm sure you've already determined the problem. We forgot to power it back up! Since the Centera was new, I had not yet added it to our Nagios monitoring system and was not paying much attention to it. I powered the cluster up and sheepishly called EMC Support to report my little flub.</p>
<p>Take-aways (don'tcha love biz-speak terms like that?):
<ul>
<li>Even experienced tech guys like me fall victim to noob shenanigans like forgetting to check power on a system before diving into troubleshooting.</li>
<li>Add systems to your monitoring solution early, even if they're not in production yet. You can always disable alerting for that particular system until it's in production, and it's a good shakedown to make sure your thresholds are reasonable. It will also tell you if you maybe shut down the system and forget to turn it back on! (Like anybody would ever do something like that...)</li>
</ul>
]]></content:encoded>
			<wfw:commentRss>http://bvlog.vosswerx.com/2009/03/09/1-is-it-turned-on/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Red light, green light</title>
		<link>http://bvlog.vosswerx.com/2008/10/27/red-light-green-light/</link>
		<comments>http://bvlog.vosswerx.com/2008/10/27/red-light-green-light/#comments</comments>
		<pubDate>Tue, 28 Oct 2008 03:06:51 +0000</pubDate>
		<dc:creator>bvoss</dc:creator>
				<category><![CDATA[funny]]></category>
		<category><![CDATA[sysadmin]]></category>

		<guid isPermaLink="false">http://bvlog.vosswerx.com/?p=213</guid>
		<description><![CDATA[One of our enterprise  application vendors recommends the following procedure for Daylight Savings Time change:
On the first Sunday in November you need to shut down blah applications during the time change to avoid duplicate chart times.
This is for a mission-critical 24x7 application that is used daily by hundreds of people.
Why don't we just go [...]]]></description>
			<content:encoded><![CDATA[<p>One of our <strong>enterprise </strong> application vendors recommends the following procedure for Daylight Savings Time change:</p>
<blockquote><p>On the first Sunday in November you need to shut down <em>blah</em> applications during the time change to avoid duplicate chart times.</p></blockquote>
<p>This is for a mission-critical 24x7 application that is used daily by hundreds of people.</p>
<p>Why don't we just go ahead and close the hospital for an hour to avoid duplicating any information?</p>
<p>Wow.</p>
]]></content:encoded>
			<wfw:commentRss>http://bvlog.vosswerx.com/2008/10/27/red-light-green-light/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Today&#8217;s post brought to you by the letter zero</title>
		<link>http://bvlog.vosswerx.com/2008/10/27/todays-post-brought-to-you-by-the-letter-zero/</link>
		<comments>http://bvlog.vosswerx.com/2008/10/27/todays-post-brought-to-you-by-the-letter-zero/#comments</comments>
		<pubDate>Mon, 27 Oct 2008 19:32:10 +0000</pubDate>
		<dc:creator>bvoss</dc:creator>
				<category><![CDATA[funny]]></category>
		<category><![CDATA[sysadmin]]></category>

		<guid isPermaLink="false">http://bvlog.vosswerx.com/?p=210</guid>
		<description><![CDATA[Me: Ok, your account has been created. Your username is blah blah blah zero.
User: Um, is that the number zero or the letter zero?
Me: (Dumbfounded silence) That would be the number zero.
User: Ok, thanks!
]]></description>
			<content:encoded><![CDATA[<p><strong>Me:</strong> Ok, your account has been created. Your username is <em>blah blah blah</em> zero.</p>
<p><strong>User:</strong> Um, is that the number zero or the letter zero?</p>
<p><strong>Me:</strong> (Dumbfounded silence) That would be the number zero.</p>
<p><strong>User:</strong> Ok, thanks!</p>
]]></content:encoded>
			<wfw:commentRss>http://bvlog.vosswerx.com/2008/10/27/todays-post-brought-to-you-by-the-letter-zero/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>
