Linux solution for ARP flux
If for some reason you have to configure multiple network interfaces in the same subnet, e.g. eth0 – 192.168.1.1/24, eth1 – 192.168.1.2/24, eth2 – 192.168.1.3/24, etc., on Linux box you’d certainly face a so called ARP flux issue.
To fix it (I assume you have kernel > 2.6.2) just set the following sysctl options as listed below:
net.ipv4.conf.eth0.arp_ignore = 1 net.ipv4.conf.eth0.arp_announce = 2
In my particular case that wasn’t a complete remedy as I had to amend an additional sysctl parameter:
net.ipv4.conf.ethX.rp_filter = 0
where ethX is your network adapter, e.g. eth0, eth1, eth3, etc.
Read more about arp_announce and arp_ignore options.
Hope that helps. Good luck.
What to do if HDvM reports unknown version
If you notice that HDvM web interface is saying that it’s running an unknown version (and also recommends to contact HDS support) or when applying a patch it fails like this:
# ./UPDATE *********************************************************************** * Update Utility * *********************************************************************** [HDvM 072003] Hitachi Device Manager Now searching for target products... The target product is not found.
Then don’t worry as it’s very easy to fix. Most probably you’re missing .HiCommandSPInfo file which by default is placed under /opt/HiCommand/. In my case, for some strange reason, I had it under /opt/ and after I copied it to the correct location (/opt/HiCommand/.HiCommandSPInfo) the issue was solved:
# ./UPDATE *********************************************************************** * Update Utility * *********************************************************************** [HDvM 072003] Hitachi Device Manager Now searching for target products... [HDvM 072000] is found.
Good luck.
Enable CIM server on MDS Cisco switch
Want to add your shiny MDS Cisco switch into HTnM (Hitachi Tuning Manager) – that’s very easy. All you have to do is to enable cim, preferably secure cim server, and configure agtw (SAN agent) using HTnM cli. How I did it…
- ssh into a switch to configure CIM server. First I copied SSL certificate (self-signed will do well) to configure secure cim server:
- Go to the server’s console of your server to do HTnM part.
- Check /opt/jp1pc/agtw/agent/mds_switch_name/jpcagt.ini to make sure that you have port 5989 and secure option enabled in it./
- Start the agent.
# copy scp://username@ssh_host/path_to_ssl_certificate_file bootflash:simserver.pem # configure t # cimserver certificate bootflash:simserver.pem # cimserver enableHttps operation cimserver enableHttps(): status: Please restart cimserver for this request to take effect # no cimserver enableHttp # no feature cimserver # feature cimserver # show cimserver status cimserver is enabled # show cimserver httpSStatus cimserver Https is enabled # show cimserver httpStatus cimserver Http is not enabled
# /opt/jp1pc/tools/jpcinssetup agtw -inst mds_swicth_name
Connection Destination ( 1:Brocade(DCFM SMI Agent) or 2:Brocade(SMI Agent for FOS) or 3:Brocade(SMI Agent for EOS) or 4:Cisco(Seed Switch) ) : 4
IP Address : ip_address
Secure Communication (Y/N) [N] : Y
Symbolic name of the fabric [mds_switch_name] :
Login ID : htnm
Login Password :
Re-enter :
Store Version [2.0] :
KAVE05080-I The instance environment is now being created. (servicekey=Switch, inst=mds_switch_name)
KAVE05081-I The instance environment has been created. (servicekey=Switch, inst=mds_switch_name)
SEED_SECURE_COMMUNICATION=Y
EOS_PORT_NUMBER=5989
# /opt/jp1pc/tools/jpcstart agtw inst=mds_switch_name KAVE06007-I The service will now start. (service=Agent Store for SANSwitch, inst=mds_switch_name) KAVE06007-I The service will now start. (service=Agent for SANSwitch, inst=mds_switch_name)
Job done. Wait for the next round of data collection and enjoy the graphs and stats from your MDS box.
RSA AM doesn’t start after kernel upgrade?
If you’ve just upgraded the kernel and your RSA AM server doesn’t start anymore and you see the following error:
com.rsa.ims.security.keymanager.sys.SystemModificationThresholdException: System was modified beyond the allowed threshold, cannot decrypt.
don’t lose consciousness since there is a simple remedy for that
# su - rsa $ cd /opt/RSAAuthenticationManager/utils/ $ ./rsautil manage-secrets -a recover $
Just enter master pasword at the prompt and you’re golden.
Migrating Hi-Track
Yesterday I was moving Hi-Tack monitor from Window to Linux and just want to give a quick overview of the process. To save all your previous Hi-Track configuration just do the following:
- Install Hi-Track, no surprisingly.
- Copy devices.export from your old installation and apply it to a new one:
/usr/hds/hitdfmon/rundfmon import ./devices.export
- Copy object.dat28 and HitDFmon.config files to /usr/hds/hitdfmon/ (that is default installation directory for Hi-Track and your could be different).
- Copy object.dat28 if you’ve configured username/password to connect to the devices you monitor.
- Start Hi-Track:
/etc/init.d/rundfmon start
Easy peasy.
Fork Yeah! The Rise and Development of illumos
Oh gosh, I truly love the way Bryan Cantrill runs his public speeches/presentation. Purely amazing, full of energy, fun and endlessness positiveness shines through out his talks. And of course, they’re always full of the info one can’t find anywhere else. Highly recommended.
Back to school
Registered for the free online classes presented by Stanford University:
I do really hope I would have enough time to keep pace with the lectures, wouldn’t brake the timings and eventually finish all the classes I’ve subscribed to.
Bye-bye google reader
After Google had introduced an updated version of the Reader and deliberately turned it into a clunky, shitty and hard on the eyes application, I decided to pack my bag and move all my feeds to www.newsblur.com. As far as I could tell this is the best Google Reader replacement and I could find. It’s semi-free, since you could only have 64 feeds without paying a cent, but the price for this fantastic service is reasonable so I didn’t hesitate a second and went with a premium account.
SystemTap
Guys, that’s not funny ay all. To tell the truth, I’ve never used SystemTap myself but always treated it as a valid option when your on Linux as there is no dtrace. But hey, read another outstanding review from Brendan Gregg and decide yourself if you could trust it. Just a short excerpt:
To write this post, my system has hung when running SystemTap more than six times (I lost count), requiring a power cycle each time. While this system seems more brittle than others on which I’ve run SystemTap (I usually have to run more than just disktop.stp to hit a crash), I still don’t feel safe using SystemTap in production.
Read the rest – it’s really worth it.
HTnM data migration between Windows and Linux
Sometime ago I was trying to migrate HTnM data, more precisely HTnM agents datastore history information, from Windows to Linux and want to share my story. Migrating HTnM configuration and its data from Windows to Linux is very easy, just follow the documentation steps verbatim. But moving historical data from the HTnM agents, whether it’s RAID or SAN, is a different thing. One may think that it’s a straightforward operation, well it really is, but the documentation doesn’t quite agree on that.
OS groups that can be migrated
OS groups are defined as follows for the Tuning Manager series. Store database migration can only be performed for the same OS groups.• Windows groups:
• Windows Server 2003 (x86)
• Windows Server 2003 (x64)
• Windows Server 2008 (x86)
• Windows Server 2008 (x64)
• HP-UX groups:
• HP-UX 11i V2 (IPF)
• HP-UX 11i V3 (IPF)
• Solaris (SPARC) groups
• Solaris 9 (SPARC)
• Solaris 10 (SPARC)
• Solaris (X64) group:
• Solaris 10 (X64)
• AIX groups:
• AIX 5L V5.3
• AIX V6.1
• Linux groups:
• Linux4 (x86)
• Linux5 (x86)
Moreover, it says that this is unsupported (I have opened the case in HDS and they confirmed my sneaking suspicions). Period.
Thankfully, I had some spare time to prove the documentation wrong. Sounds over-confide, I know. ;-)
Anyway, I tried to pretend that I hadn’t read that part that stated that this was unsupported and went through the steps as I was migrating data between Windows-based machines. First, I boldly tried to migrate directly from Windows (HTnM 6.4) to Linux (HTnM 7.1) and everything went very nice until I tried to restore the data from RAID agent on Linux server. When I ran jpcrestor tool I was told that data model of the agent on the migration destination (Linux server) differed from the migration source (Windows) version. JFYI, HTnN ver. 6.4 uses data model ver 7.6, whilst HTnM ver. 7.1 has data model ver. 8.0. For me it meant that before being able to migrate the store database I had to convert from older to newer version. Hitachi provides a special tool just for that:
# /opt/jp1pc/tools/jpcdbctrl dmconvert -d /tmp/export/ KAVE05847-I Data model conversion of backup data will now start. (dir=/tmp/export/) KAVE05099-E An internal command terminated abnormally. (/opt/jp1pc/bin/jpccvtmdl -sdict "/tmp/export//STDICT.DAT.bak" -ddict "/tmp/export//STDICT.DAT" -stdir "/tmp/export/" -wkdir "/tmp/export//cvtwork" -directdbdir "/tmp/export/") KAVE05849-E Data model conversion of backup data ended abnormally. (dir=/tmp/export/)
As I mentioned before, I opened a case with HDS and was “pleased” to hear that what I was doing was unsupported. So I was on my own. But I didn’t give up. I’d decided to upgraded HTnM on Windows to version 7.1 and repeat everything once again. To my surprise, since I didn’t have to mess around with converting data model to the same format, agent’s data were restored swimmingly. Important note. I had to run dos2unix against jpcsto.ini file since it contained Windows CRLF line terminators which is expected since I was migrating from Windows server. Once I did that, there were no more hurdles to overcome:
# ./jpcresto agtd /tmp/hdsb inst=uspv-array Directory receiving backup data = /opt/jp1pc/agtd/store/uspv-array Directory supplying backup data = /tmp/hdsb Clearing indexes... Appropriate indexes removed. Replace in progress. Replacing /tmp/hdsb/STDC.DB to /opt/jp1pc/agtd/store/uspv-array/STDC.DB ... Replacing /tmp/hdsb/STDC.IDX to /opt/jp1pc/agtd/store/uspv-array/STDC.IDX ... Replacing /tmp/hdsb/STDM.DB to /opt/jp1pc/agtd/store/uspv-array/STDM.DB ... Replacing /tmp/hdsb/STDM.IDX to /opt/jp1pc/agtd/store/uspv-array/STDM.IDX ... Replacing /tmp/hdsb/STAM.DB to /opt/jp1pc/agtd/store/uspv-array/STAM.DB ... Replacing /tmp/hdsb/STAM.IDX to /opt/jp1pc/agtd/store/uspv-array/STAM.IDX ... Replacing /tmp/hdsb/STPI/5/2010/1130/001/PI_LDS.DB to /opt/jp1pc/agtd/store/uspv-array/STPI/5/2010/1130/001/PI_LDS.DB ... Replacing /tmp/hdsb/STPI/5/2010/1130/001/PI_LDS.IDX to /opt/jp1pc/agtd/store/uspv-array/STPI/5/2010/1130/001/PI_LDS.IDX ... Replacing /tmp/hdsb/STPI/5/2010/1130/001/PI_CLPS.DB to /opt/jp1pc/agtd/store/uspv-array/STPI/5/2010/1130/001/PI_CLPS.DB ... Replacing /tmp/hdsb/STPI/5/2010/1130/001/PI_CLPS.IDX to /opt/jp1pc/agtd/store/uspv-array/STPI/5/2010/1130/001/PI_CLPS.IDX ... Replacing /tmp/hdsb/STPI/5/2010/1130/001/PI_LDE.DB to /opt/jp1pc/agtd/store/uspv-array/STPI/5/2010/1130/001/PI_LDE.DB ... Replacing /tmp/hdsb/STPI/5/2010/1130/001/PI_LDE.IDX to /opt/jp1pc/agtd/store/uspv-array/STPI/5/2010/1130/001/PI_LDE.IDX ... Replacing /tmp/hdsb/STPI/5/2010/1130/001/PI_PTS.DB to /opt/jp1pc/agtd/store/uspv-array/STPI/5/2010/1130/001/PI_PTS.DB ... ... SKIPPED ... Replacing /tmp/hdsb/STPD/2011/0726/001/PD_RGC.DB to /opt/jp1pc/agtd/store/uspv-array/STPD/2011/0726/001/PD_RGC.DB ... Replacing /tmp/hdsb/STPD/2011/0726/001/PD_RGC.IDX to /opt/jp1pc/agtd/store/uspv-array/STPD/2011/0726/001/PD_RGC.IDX ... Replacing /tmp/hdsb/STPH.DB to /opt/jp1pc/agtd/store/uspv-array/STPH.DB ... Replacing /tmp/hdsb/STPH.IDX to /opt/jp1pc/agtd/store/uspv-array/STPH.IDX ... Replacing /tmp/hdsb/STPA.DB to /opt/jp1pc/agtd/store/uspv-array/STPA.DB ... Replacing /tmp/hdsb/STPA.IDX to /opt/jp1pc/agtd/store/uspv-array/STPA.IDX ... Replacing /tmp/hdsb/STPT.DB to /opt/jp1pc/agtd/store/uspv-array/STPT.DB ... Replacing /tmp/hdsb/STPT.IDX to /opt/jp1pc/agtd/store/uspv-array/STPT.IDX ... KAVE06006-I Restore processing of the Store database terminated normally.
More than that, when I logged onto HTnM web interface and searched my one year old data from USP-V array – they were all their for my observation. Sweet. What’s the gist of this post, you may ask. Well, not only don’t ever give up but always take every recommendation, even if it comes from a vendor’s support team, with a grain of salt. Peace.