diff --git a/pages/DallasPoP.md b/pages/DallasPoP.md index e69de29..31245c7 100644 --- a/pages/DallasPoP.md +++ b/pages/DallasPoP.md @@ -0,0 +1,379 @@ +# Dallas Google PoP + +### Old Google PoP + +We're in Google POP in Via West + +``` +Viawest/Dataside +1950 Stemmons Freeway, Suite 2033 +Dallas TX 75027 ++1-214-231-0162nfs0 +Mark for "inside delivery" + +Google PoP DFW06 +Rack Location: 6.3 +Rack: ESW E-00589-01, Black Caster and Mounting Rails, No Door, 7Ft. +``` + +### New Google POP + +``` +Site DFW 28 +Cage 4000.40 +1950 N Stemmons Freeway +Suite 4000 +Dallas, TX 75207 + +Racks 4-11 4-12 + RGnet in 4-12 + IETF/Deadalus/RGnet in 4-11 + +Shipping Address: + Google - chris morrow + c/o: Infomart Suite 4000 + 1950 N Stemmons Freeway + Dallas, TX 75207 +``` + + + +## Rack Physical and Wiring + +| hostname | switch | ipmi | oob | oobname | pwr | rack | | +| :--------------------------- | ------------------------------------------------ | ------------ | ------- | ---------- | ------------------------------------------------ | ----- | ------------------------------------- | +| o2.dfw.rg.net | sw99-42 | | 1.8 | o2 | | | Cyclades® ACS Advanced Console Server | +| r0.dfw.rg.net-rp0 | 0.45 | | 0.01 | r00 | subfl | 03-13 | | +| r0.dfw.rg.net-rp0mgt | 0.33 | | | | | 03-13 | | +| r0.dfw.rg.net-rp1 | 0.?? | | 0.02 | r01 | | 03-13 | | +| r0.dfw.rg.net-rp1mgt | 0.34 | | | | | 03-13 | | +| [r0.2/0/4 LAN] | 0.04 | | | | | 03-13 | | +| [r0.2/0/5 ntt fiber] | 0.05 | | | | | 03-13 | | +| [r0.2/0/6 spare] | 0.06 | | | | | 03-13 | | +| [r0.2/0/7 equinix ix] | 0.15 | | | | | 03-13 | | +| sw0.dfw.rg.net | | | 0.13 | sw0 | [(1)](https://wiki.rg.net/wiki/DallasPoP#point1) | 17 | | +| [equnix IX fiber] | 0.46 | | | | | | | +| r1.dfw.rg.net.2/0 | 0.07 | | | r1 | | 18-21 | | +| r1.dfw.rg.net.2/1 | 0.08 | | | | | 18-21 | | +| r1.dfw.rg.net.2/2 | 0.09 | | | | | 18-21 | | +| r1.dfw.rg.net.2/3 | 0.10 | | | | | 18-21 | | +| r1.dfw.rg.net.mgt | 0.43 | | | | | 18-21 | | +| o1.dfw.rg.net | 0.40 | | 0.aux | | 1.1 | 22 | | +| panduit cover-up | | | | | | 23 | | +| o0.dfw.rg.net | 0.44 | | 1.aux | | 0.1 | 24 | | +| pwr1.dfw.rg.net | 0.42 | o2.9 / o2.10 | 0.15 | pwr1 | subfl | 26 | | +| pwr0-blue.dfw.rg.net | 0.18 | | o2.16 | pwr0-blue | subfl | - | | +| raid0.dfw.rg.net | 0.12 | 0.26 | 0.04 | nfs0 | 0.5/1.5 | 31-33 | | +| dfw0.psg.com | 0.11 | 0.31 | 0.03 | dfw0 | 0.3/1.3 | 34 | | +| sw1.dfw.rg.net | 0.46 | | 1.01 | sw1 | [^2] | 35 | ??? | +| vm0.dfw.rg.net | [^1] | 1.22 | 1.10 | vm0 | 0.4/1.4 | 35-36 | | +| vm1.dfw.rg.net | [^1] | 1.23 | o2.41 | vm1 | 0.4/1.4 | 35-36 | | +| vm2.dfw.rg.net | [^1] | 1.24 | o2.42 | vm2 | 0.4/1.4 | 35-36 | | +| vm3.dfw.rg.net | [^1] | 1.25 | o2.43 | vm3 | 0.4/1.4 | 35-36 | | +| kingsx.richosman.com | 0.21 | | o2.48 | kingsx | 0.6/1.6 | 37 | | +| b0.noc.ietf.org | | xx | o2.45 | b0 | xx | 99 | | +| paddington.richosman | 0.22 | | o2.47 | paddington | 0.8/1.8 | 37 | | +| nfs0.dfw.rg.net | 0.12 | 0.24 | 0.27&29 | nfs0 | 0.5/1.5 | 42-43 | | +| dallas01.mux.peering.usc.edu | 0.37/0.39 | | o2.46 | usc | 0.7/1.7 | 44 | | + + + +## vm0/1/2/3.dfw.rg.net Ganeti Cluster / 198.180.152.30/31/32/33 + +VMs are [Supermicro X8DTT-HF Motherboards](http://www.supermicro.com/products/motherboard/QPI/5500/X8DTT-H.cfm?INF=) with 48G of RAM each + in [​827H-R1200B Chassis](http://www.supermicro.com/products/chassis/2U/827/SC827H-R1200.cfm) + or [​SC827H-R1400B Chassis](http://www.supermicro.com/products/chassis/2U/827/SC827H-R1400.cfm) + 12 SATA-3 250G Drives model WDC WD2502ABYS-0. + +[^1]: vm servers' ethers are connected to sw2.dfw as follows: + +| vm | ipmi | e0 | e1 | +| ---- | ------ | ------ | ------ | +| vm0 | sw0-13 | sw0-19 | sw2-43 | +| vm1 | sw0-14 | sw0-20 | sw2-44 | +| vm2 | sw0-15 | sw0-21 | sw2-45 | +| vm3 | sw0-16 | sw0-22 | sw2-45 | + +[^2]: PTSS/B is connected to power 0.2/1.2 and controls both sw0 and sw1 + +### VM Node Layout (from rear, rack DFW28-04-12) + +| | | +| ---- | ---- | +| vm0 | vm2 | +| vm1 | vm3 | +| | | + + + +### vm[0-3].dfw.rg.net Disk Drives + +| Bay | vm | Model | Serial | Size | Date | | +| ---- | ---- | ------------------ | ------------ | ---- | ---------- | ----------------------------- | +| A0 | vm3 | WD2000F9YZ | WMC5C0D3HDAV | 2TB | 2015.03.16 | | +| A1 | vm3 | WD2000F9YZ | WMC5C0D3TV0R | 2TB | 2015.03.16 | | +| A2 | vm3 | WD2000F9YZ-09N20L1 | WX11D363TV6F | 6TB | 2018.01.10 | was WMC5C0D5KYPP | +| B0 | vm2 | WD2000F9YZ | WMC5C0D2NV6N | 2TB | 2015.03.16 | | +| B1 | vm2 | WD2005FBYZ | WMC6N0M3AEHR | 2TB | 2015.03.16 | WMC5C0D623ZS pulled 2018.11.3 | +| B2 | vm2 | WD2000F9YZ | WMC5C0D75CRL | 2TB | 2015.03.16 | | +| C0 | vm1 | WD2000F9YZ | WMC5C0D61JYX | 2TB | 2015.03.16 | | +| C1 | vm1 | WD2000F9YZ | WMC5C0DA1006 | 2TB | 2015.03.16 | | +| C2 | vm1 | WD2000F9YZ | WMC6N0H8FVYN | 2TB | 2015.03.16 | was WMC5C0D44TEU | +| D0 | vm0 | WD2000F9YZ | WMC5C0D2APXE | 2TB | 2015.03.16 | | +| D1 | vm0 | WD2000F9YZ | WMC5C0D34PT0 | 2TB | 2015.03.16 | | +| D2 | vm0 | WD2000F9YZ | WMC5C0D2HXNV | 2TB | 2015.03.16 | | + + + +### Ganeti Cluster / 198.180.152.30/31/32/33 + +``` +vm0.dfw.rg.net:/root# ./do-stat +Node DTotal DFree MTotal MNode MFree Pinst Sinst +vm0.dfw.rg.net 3.6T 3.3T 47.3G 33.9G 39.2G 3 2 +vm1.dfw.rg.net 3.6T 3.3T 47.3G 21.4G 39.4G 3 3 +vm2.dfw.rg.net 3.6T 3.5T 47.3G 24.4G 42.7G 3 1 +vm3.dfw.rg.net 3.6T 3.0T 47.3G 22.3G 32.2G 4 2 +``` + +### Hosts on vm0/1/2/3.dfw.rg.net + +| Instance | Primary_node | Secondary_Node | MaxMem | DiskUsage | IP_Address | SysAdmin | +| ------------------- | -------------- | -------------- | ------ | --------- | --------------- | ------------------------------------------ | +| ca0.rpki.net | vm0.dfw.rg.net | vm2.dfw.rg.net | 2.0G | 128.1G | 198.180.152.16 | Randy Bush | +| ca.rg.net | vm0.dfw.rg.net | vm3.dfw.rg.net | 4.0G | 128.0G | 198.180.152.6 | Randy Bush | +| cu0.dfw.rg.net | vm2.dfw.rg.net | vm3.dfw.rg.net | 512M | 12.0G | 198.180.152.71 | Steven M Bellovin John Koh | +| git.rg.net | vm3.dfw.rg.net | vm2.dfw.rg.net | 2.0G | 34.0G | 198.180.152.8 | Randy Bush | +| icir.dfw.rg.net | vm3.dfw.rg.net | vm2.dfw.rg.net | 1.0G | 32.0G | 198.180.152.57 | Bill Marczak | +| netsrv.dfw.rg.net | vm3.dfw.rg.net | | 1.0G | 100G | 198.180.152.15 | Randy Bush | +| nms.rg.net | vm3.dfw.rg.net | vm0.dfw.rg.net | 2.0G | 20G | 198.180.152.9 | Randy Bush | +| petunia.hactrn.net | vm2.dfw.rg,net | | 6G | 120G | 198.180.152.200 | Rob Austein | +| pfs0.dfw.rg.net | vm0.dfw.rg.net | vm2.dfw.rg.net | 512M | 8.1G | 198.180.152.62 | Randy Bush | +| rb-build.dfw.rg.net | vm2.dfw.rg.net | vm1.dfw.rg.net | 16G | 150.1G | 198.180.152.62 | Randy Bush | +| ymbk.com | vm2.dfw.rg.net | vm3.dfw.rg.net | 512M | 16.1G | 198.180.152.10 | Randy Bush | + + + + +### nfs0.dfw.rg.net + +[PDSME+](http://www.supermicro.com/products/motherboard/Xeon3000/3010/PDSME_.cfm) + [​AOC-SIM1U+ IPMI Card](http://www.supermicro.com/products/accessories/addon/SIM.cfm) + +### nfs0.dfw.rg.net Disk Drives + +``` +for i in 0 1 2 3 4 5 6 7; do tw_cli /c2/p$i show serial; done +``` + +| Port | Model | Serial | Size | Date | +| -------- | ------------------ | --------------- | ---- | ------------------ | +| p0 | ST4000DM000-1F2168 | Z300L98C | 4TB | don't know | +| p1 | WD4000F9YZ-09N20L1 | WD-WCC5D1DS4012 | 4TB | 2019.04.07 | +| p2 | WD60EFRX-68L0BN1 | WD-WX41DA57H7UJ | 6TB | 2018.03.09 | +| p3 | WD4000F9YZ-09N20L1 | WD-WCC5D2KHHYE8 | 4TB | 2016.06.09 | +| p4 | ST4000DM000-1F2168 | Z300KBMM | 4TB | don't know | +| p5 | ST4000DM000-1F2168 | Z300KBQN | 4TB | don't know | +| p6 | ST4000DM000-1F2168 | Z300L8J5 | 4TB | don't know | +| p7 | WD4000F9YZ-09N20L1 | WD-WMC5D0D7V0L9 | 4TB | 2015.12.13 | +| | | | | | +| p1 | ST4000DM000-1F2168 | Z300FKES | 4TB | removed 2019.04.07 | +| p2 | ST4000DM000-1F2168 | Z300L9T3 | 4TB | removed 2018.03.09 | +| incoming | HUS724040ALS640 | PBK4K70X | 4TB | 2015.12.18 | + + + Physical layout is + +| 4 | 5 | 6 | 7 | +| ---- | ---- | ---- | ---- | +| 0 | 1 | 2 | 3 | + +### raid0.dfw.rg.net + +[Mobo Supermicro X7SB4/E](https://www.supermicro.com/products/motherboard/Xeon3000/3210/X7SB4.cfm) [IPMI AOC-IPMI20-E](http://www.supermicro.com/products/accessories/addon/AOC-IPMI20-E.cfm) + +raid0 disk drive physical order markings on drive carriers + +| 00 | 10 | 20 | 30 | +| ---- | ---- | ---- | ---- | +| 01 | 11 | 21 | 31 | +| 02 | 12 | 22 | 32 | +| 03 | 13 | 23 | 33 | + +``` +for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16; do + echo -n "1/$i "; + smartctl -i -d hpt,1/$i /dev/hptrr | grep ^Serial; + done +``` + +| Bay | ID | Drive | Name | Serial Number | +| ---- | ---- | ----- | --------- | --------------- | +| 20 | 1/1 | da0 | JBOD-1-1 | WD-WMAZA9378538 | +| 21 | 1/2 | da1 | JBOD-1-2 | MK0171YFKMA4AG | +| 22 | 1/3 | da2 | JBOD-1-3 | WD-WCC300755734 | +| 23 | 1/4 | da3 | JBOD-1-4 | WD-WMAZA9520754 | +| 00 | 1/5 | da4 | JBOD-1-5 | WD-WCAVY0114154 | +| 01 | 1/6 | da5 | JBOD-1-6 | WD-WMC5C0D9L6HV | +| 02 | 1/7 | da6 | JBOD-1-7 | WD-WMC6N0H8K48M | +| 03 | 1/8 | da7 | JBOD-1-8 | WD-WCC300759632 | +| 30 | 1/9 | da8 | JBOD-1-9 | WD-WCAVY0111166 | +| 31 | 1/10 | da9 | JBOD-1/10 | WD-WCAVY0117958 | +| 32 | 1/11 | da10 | JBOD-1/11 | WD-WMC5C0D8Z7FV | +| 33 | 1/12 | da11 | JBOD-1/12 | WD-WMC6N0J6S9AV | +| 10 | 1/13 | da12 | JBOD-1/13 | MK0171YFKMA59G | +| 11 | 1/14 | da13 | JBOD-1/14 | WD-WMC6N0F8FU0U | +| 12 | 1/15 | da14 | JBOD-1/15 | WD-WCAVY0090692 | +| 13 | 1/16 | da15 | JBOD-1/16 | WD-WCC4M5VJ4UK3 | + +da10 WD-WCAVY0127426 replaced 2016.09.15 + da1 S2F06GX5 replaced 2016.01.2 + da2 WD-WCAVY0112831 replaced 2014.05.12 + da5 WD-WCAVY0107135 replaced 2017.03.25 + da13 W1E0B4NJ replaced 2017.06.xx + da6 WD-WCAVY0127865 replaced 2018.01.07 + +Spare Drives + +| | | | +| ---- | ------------- | ------------ | +| Size | Serial Number | Model # | +| 2tb | WMC6NOF8FU0U | WD2005FBYZSP | +| 2th | WMC6NOF2KZU6 | WD2005FBYZSP | +| | | | + +### dfw0.psg.com + +dfw0 drives `for i in 4 5 6 7; do echo /dev/ad$i; smartctl -a /dev/ad$i | egrep 'Model|Serial'; done` + +| | | | | +| ---- | ------------------ | ------------ | ---- | +| Bay | Model | Serial | | +| | | | | +| ada0 | ST31000340NS | 9QJ3Q29E | 1TB | +| ada1 | WD20EADS-00R6B0 | WCAVY0388932 | 2TB | +| ada2 | WD2005FBYZ-01YCBB1 | WMC6N0F2KZU6 | 2TB | +| ada3 | ST31000340NS | 9QJ3M5N4 | 1TB | +| | | | | + +dfw0 disk drive physical order + +| ada4 | ada6 | ada5 | ada7 | +| ---- | ---- | ---- | ---- | +| ada0 | ada2 | ada1 | ada3 | + +logical drive labels + +| label/disk00 | ada0s3 | +| ------------ | ------ | +| label/disk01 | ada1s3 | +| label/disk02 | ada2s1 | +| label/disk03 | ada3s1 | + +## Disgusting Details + +``` +r0 cisco 12406 +r1 Juniper MX240 +sw0 cisco 2960G-48TC-L with fiber to 12406 +sw1 Juniper EX2200-48T-4G +dfw0 1ru compute server SuperMicro SYS-6015W-NTRB w/ X7DWU +dfw1 3ru big raid server SuperMicro CSE-836TQ-R800B w/ X7SBE +dfw2 2ru Junosphere virtualization server +o0 cisco 2511 with copper to sw0 and serial cables to all craft ports +o1 cisco 2511 with copper to sw0 and serial cables to all craft ports +o2 Cyclades® ACS Advanced Console Server +pwr0 Servertech Horizontal Power Strip (8port) Part Number: CW-8H2-C20 +pwr1 Servertech Horizontal Power Strip (8port) Part Number: CW-8H2-C20 + +--- + +Power + SO1-A:5C2.2-62/64 + SO2-A:5C2.2-66/68 + SO1-B:5D2.2-62/64 + SO2-B:5D2.2-66/68 + C-Side is Blue and that = "A" feed + D-Side is Yellow and that = "B" feed + [ SO2-B:5D2.2-66/68 will not be used ] + +The demarc for the xconnects is as follows: + pp01.dfw06 + 137-138 M3 Future + 139-140 M4 Future + 141-142 M5 Equinix IX Cable ID #20466027 Terminated on CP:0104:1068601 ports 3/4 + 143-144 M6 NTT/Verio r08.dllstx09.us.bb g0/6/0/29 + +Rge Equinix order info for the exchange fiber: +Ordering contact- Chris Morrow and/or CSM- Shazyra Irizarry +Technical contact – Randy Bush +IBX- DA1 +Order: 1-97362053795 +WOA- 3-97445043133 (IX Port Install) +WOA- 3-97448754941 (NCC Cable install) +Serial # 20466027 ; terminated on CP:0104:1068601 ports 3/4 + +--- + +Dallas Equinix Colo, Downstairs + +Google demarc for the riser cable on the Equinix end is Suite 1034 Cage +00372 Cabinet 105 + +NTT/Verio TDI +Cage 220, Cab 101 FP#1, Port NA, SMF/SC. +r08.dllstx09.us.bb g0/6/0/29 + +Media: MMF +Connector: SC + +--- ntt circuit --- + +EQUINIX INC. - Cross Connect Order Complete - 817885.1.1.2 +Completed On: 08-JUN-2009 +IBX: DA1 +Type of Order: Cross Connect +Cage: DA1::00080:EQUINIX.COM +Cabinet Number: 0101 +A-Side Patch Panel/Module/Port: PP:0105:2288||1 +Z-Side Customer and Cage Number: YOUTUBE, LLC + DA1::00372:YOUTUBE +Z-Side Cabinet Number: 0105 +Carrier Circuit ID(if applicable): EQUINIX SANDBOX +Z-Side Patch Panel/Module/Port: VP:0105:69862||143 +Cable ID: 1234912 + +EQUINIX INC. - Cross Connect Order Complete - 817885.2.1.2 +Completed On: 05-JUN-2009 +IBX: DA1 +Type of Order: Cross Connect +Cage: DA1::00080:EQUINIX.COM +Cabinet Number: 0101 +A-Side Patch Panel/Module/Port: PP:0105:74459||3 +Z-Side Customer and Cage Number: NTT AMERICA, INC. + DA1::00220:NTT AMERICA, INC. (Verio) +Z-Side Cabinet Number: 0101 +Carrier Circuit ID(if applicable): EQUINIX SANDBOX +Z-Side Patch Panel/Module/Port: PP:0101:2375||15 +Cable ID: 1234913 + +For both crossconnects, even though the "Cabinet Number" field shows +"0101", the actual Equinix A-Side cab is "0105" as listed in the panel +info. And to reduce cabling and points of failure, the IBX site +technicians said that they would be connecting both crossconnects +directly into each other rather than with an intermediate patch cable. + +--- equinix exchange --- + +csw7.da1, gig 0/35, Sandbox/RGNet, IX, circuit id 1236454 +``` + + + +## Routers + +[![120131.dallas-routers.jpg](../pics/120131.dallas-routers.jpg)] + +## Tapping BGP Feeds + +[![110617.dfw-data-tap.jpg](../pics/110617.dfw-data-tap.jpg)] \ No newline at end of file