More Memory

posted Apr 9, 2017, 5:22 PM by ms b0b

Ordered 8x 8GB PC3-12800 ECC registered and installed them in the virtualization server. Sounds simple, but it was not without drama.

First ordered the memory from one vendor around mid-March. The tracking info never got updated. Notified the vendor after the delivery window and promptly received refund. Too bad the courier dropped the ball, but the vendor made me whole.

Ordered the memory from a different vendor on April 4, received them on April 7. Nice.

Why 8GB modules? DDR3 ECC registered memory comes in 4 densities: 2GB, 4GB, 8GB and 16GB. PowerEdge R510s have 8 memory slots (4 per processor.)

2GB modules can't achieve meaningful total memory size so not even under consideration.

4GB modules are dirt cheap. About $5 per module if you look around. This density is fine for systems with lots of memory slots like R710, but a R510 can only have maximum of 32GB.

8GB modules cost about $20 per module, but I was able to find them at $15. R510 can achieve 64GB total memory which is more than I need for now. Consider 8GB unbuffered non-ECC DDR3 or DDR4 RAM for desktops are at about $50 to $60 per module, this is still a bargain.

16GB modules are too expensive. 

Now the lineup looks like:

Virtualization server
2x Xeon E5620 Quad Core 2.4GHz
8x 8GB DDR3 ECC Registered (64GB total)
ESXi 6.5

1x Xeon E5630 Quad Core 2.53GHz
4x 2GB DDR3 ECC Registered (8GB total)
1x 128GB SATA SSD for L2ARC
1x 64GB SATA SSD for ZIL

Spare R510
1x Xeon E5630 Quad Core 2.53GHz
4x 4GB DDR3 (16GB total)

Spare parts
4x 4GB DDR3 ECC Registered

To put the spare memory parts into service, I would need to get additional processor for the spare R510. Maybe I will do that next...

New Storage Hardware, Part 2

posted Mar 24, 2017, 3:40 PM by ms b0b

As alluded in the New Storage Hardware, MD1200 has fans that are almost too loud to bear. Searching for a solution again.

Option 1: Replace the Delta PFC0812DE fans with something that spins slower like the 6000rpm model.
Drawbacks: Substitute fans are pricy. Slower fans will have less airflow. Anecdote of MD1200 throwing faults if the fan is too slow. The SC440 and the MD1200 take up 4U and 2U respectively.

The clincher is not the technical issues. Rather, it is how much rack space the SC440 and the MD1200 are occupying. Good thing I have another R510 with 8 drive bays. Before we jump into it, there is another hurdle.

1x Xeon E5630

The PERC 6/iR's LSI 1078E controller does not support IT mode. Even if I flash it to LSI firmware, IT mode will not be available. I spent 1 week researching my options.

There are plenty of people who replaced the PERC 6/iR with the PERC H200 flashed to LSI IT mode firmware. I can do that too, but the PERC 6/iR uses SFF-8484 "wide" connectors and the H200 uses SFF-8087 connectors. I would need to replace the cables, and they don't come cheap.

Looking for ways to reuse the SFF-8484 cables led me to the SAS 6/iR with the LSI 1068E controller. The LSI 1068E is compliant to the SAS 3Gbps standard which has 2.2TB limit. My array has 2TB drives. Perfect.

The SAS 6/iR is cheap. Like $6 cheap on eBay. Googling for cross flashing the equivalent LSI 3081E-R IT mode firmware on the SAS 6/iR turned up exactly one result. Given how inexpensive the part was, it was worth the risk. I won't bore you with the technical details but the flashing was successful.

Moved the HDDs and SSDs from MD1200 to R510's hot swap bays and the USB flash drive from the SC440 to R510's internal USB port. Turned on the R510 and reconfigured the network, and FreeNAS is ready to serve again.

The R510 is much quieter than the MD1200. iDRAC6 reports power usage between 120 and 

New Storage II
R510 8 LFF bay chassis
1x E5630
SAS 6/iR flashed to LSI 3081E-R IT mode firmware
6x 2TB HDD
2x SSD for L2ARC and ZIL

The MD1200 and LSI 9200-8e will standby until I outgrow R510's 8 drive bays.

New Storage Hardware

posted Mar 22, 2017, 9:06 AM by ms b0b   [ updated Mar 22, 2017, 9:13 AM ]

Once the virtualization host role got moved off the PowerEdge SC440, it is freed up for other tasks. Naturally, it seemed to make sense to move FreeNAS off the Core 2 Duo E4400 white box to the PowerEdge SC440.

First, a little background information. All of these servers are booting off USB flash drives. I have been using the miniscule SanDisk Cruzer Fit 16GB for the last three years with great success. It makes moving roles around simple and allows me to dedicate the entire disk to ZFS.

To prepare the SC440 for the storage role, I added a LSI SAS 9200-8e HBA and a Broadcom 5709 dual port gigabit NIC to it. Updated the 9200-8e HBA to the latest P21 IT mode firmware because of FreeNAS and ZFS. The 6x 2TB SATA disks go into a PowerVault MD1200 directly attached storage (DAS).

And the first start up. SC440 boots into FreeNAS. The ZFS pool is mounted. All services are available. Look good, but holy shit the MD1200 is loud. The Delta PFC0812DE fans in each power supply just wails. I need to do something about that...

Salvaged some SSD from desktops that were going to be decommissioned. Assigned one each to L2ARC and ZIL role.

PowerEdge SC440
Core 2 Quad Q6700 2.66GHz
LSI 9200-8e
MD1200 with 6x 2TB SATA disks

New VM Host

posted Mar 21, 2017, 9:36 AM by ms b0b

Among the equipment I received are two Dell PowerEdge R510 with single processor each. I combined the processor and RAM into one system and ended up with a new virtualization host:

PowerEdge R510
2x Xeon E5620

I put the free ESXi vSphere 6.5 on this host in order to run the GNS3 simulator.

Starting Point

posted Mar 21, 2017, 9:10 AM by ms b0b

In January, I received some decommissioned hardware from my employer, and I have been making incremental changes since then. To best document the improvements, it's important to frame the work by defining the starting configuration.

PowerEdge SC440
Core 2 Quad Q6700 2.66GHz
XenServer 7

Core 2 Duo E4400 2GHz
6x 2TB 7200 RPM SATA
FreeNAS 9.2

TP-Link Archer C9

They are former desktops that got a second life as servers.


posted Mar 20, 2017, 10:10 PM by ms b0b

Time for a fresh start with new contents and new challenges.

Move older contents into the Archive area. It is not accessible through the navigation bar so you will need go to the Sitemap first.

1-6 of 6