Tales from the Towers Chapter 57: 2016 – The Year of Galactus

It’s time for the battle to begin, Galactus is entering the war. But a war isn’t fought on the battlefield alone, it’s also fought in the strategy room to decide which weapons to build and use, to train soldiers to use those weapons, deploy resources, and bring them to bear on the enemy: copper wires and over-priced, tax-subsidized fiber. Look, if we can have a War on Drugs, War on Poverty, War on Terror, War on Soda, War on HoHos (the government crossed the line on this one, time for a Constitutional Amendment protecting HoHos) in the school lunch cafeteria room and so on, we can have a War on Incumbent monopolized over-regulated Internet (as evidenced by T-Tommy Wheeler not even acknowledging fixed wireless in his latest report). Anyone who disagrees should just read the tax and fee part of their latest telecom bill and take some Pepto Bismol to avoid hurling.

The first part of Galactus, Destroyer of Wired Worlds, was simply getting the backhaul upgraded enough to handle the speeds we expect to deliver at APs. Add to that, we can’t use 5GHz for backhaul because we are going to need all the 5Ghz spectrum available for customers. That’s harder than it sounds when you need 400Mbps or more to feed the AP and you want to stay below $5K per location per link.

Since we aren’t using a tower-based model and we have a lot of APs to deploy, the cost of backhaul to each AP is critical. To that end we looked at the Ubiquiti AF24 ($3K) per link and IgniteNet MetroLinq 60GHz ($1100) which puts us on the bleeding edge. With IgniteNet, we are still suffering like a kid with a nosebleed as they haven’t yet shipped a product (as far as I know). We are currently changing deployment strategies to give IgniteNet time to get to market since their price/performance numbers are as enticing as a GoDaddy commercial during the SuperBowl (as I’m writing this, I got an email that shipments are starting to go out). In the meantime, we installed eight AF24 links at distances up to 2 miles and moved around our Mimosa B5 units to clean up the 5GHz band to avoid the APs. Fortunately for us, we have a co-location customer who is letting us put three AF24 radios and with fourth on the way along with a sector. Santa is going to get radiation burns trying to get down the chimney. Since the AF24 isn’t as cute as our typical CPE, this is quite an achievement to get past the missus.

Licensed won’t work on the client AP links in this model due to costs and other factors, so for the moment, it’s not an option. Although the Mimosa B11 is the only one even close to financially attractive for this model, the FCC rules require a minimum two-foot antenna and putting that on a suburban house is a problem, especially if you need two or three of them on the same house. We temporarily put up a three-foot antenna with a Mimosa B5 on a house and wrapped it with Christmas lights so the neighbors think it’s a decoration. That buys us some time through Christmas anyway.

For 24Ghz radios, we looked at three different systems: Ubiquiti AF24, SAF, and Exalt. The SAF Integra and Exalt products come with higher prices and longer warranties than the AF24. The warranty was an issue with one of our clients until I discovered that you could buy an AF24 on Amazon. And here is a big secret which is going to really tick off distributors and manufacturers who haven’t figured it out yet: if you buy it off Amazon, you can get a 4-year warranty for $125 more for the AF24s. This applies to pretty much any radio purchased off Amazon. My suggestion to distributors: talk to SquareTrade or Warrantech and get with the program. These warranties even cover accidentally dropping the equipment off a tower. Although a 40-foot drop would probably turn an AF24 with its plastic case into confetti (and no we didn’t test this although there are pieces of a Powerbridge in my backroom that did a 40-foot slam test so I’m extrapolating. What’s amazing is that the PTP600 right next to it went back up on the new pole and is working great). It’s still nice to know someone else would pay to replace it. If you are going to spend $1500 for a radio, it’s probably the best $125 spent unless you think every single AF24 you put up is going to last that long.

Ahh, but the radio isn’t the only problem. We still need a switch our router to hook the AF24s and the other radios on the house/facility with PoE. At minimum, a Netonix WS-6-port mini is pretty much the smallest switch you can get with PoE that can handle the 50W load of an AF24. Netonix should have power supplies for this, but you might have to search for the bigger ones. If you go bigger though, like a WS-12 and you plan on running a couple of AF24s (we have one location with three of them) using the PoE ports on the switch, you will need 150-250W power supplies. We are going with batteries and 48V digital chargers in our NEMA box.

One problem with this model based on APs on houses or other low-level structures is that LOS for 24GHz or 60GHz backhauls is difficult to do. Picking locations for APs sometimes has to be a compromise between the best location for the AP and LOS for the backhauls. That might mean more backhauls or more APs but the benefits of short range to the clients far outweigh the extra costs. The ROI of 30 users paying $40 per month is fairly short (adjust prices accordingly), e.g. 4-5 months, and we expect to exceed that. Vivint is using this model with 28GHz licensed spectrum and using cell towers but the costs are higher and the Cambridge equipment they are using is limited today to about 300Mbps full-duplex at the AP side. Galactus is a hungry beast and devours bandwidth like I devour HoHos, and based on some testing, we will need lots of it. Since we need 400Mbps per AP minimum to kick the copper providers’ teeth in, puny 300Mbps PTMP radios aren’t going to make it. There currently isn’t any PTMP equipment out there yet that can feed Galactus so you need to plan for a lot of PTP links.

Now that we have enough bandwidth to download the Library of Congress before we even start deploying the rest of Galactus, we did some testing on the Ubiquiti APs that are getting replaced. Most of our Rocket 5Ms are original XM radios, so basically, 802.11n with underpowered processors and AirMax overhead. Even with those APs, user bandwidth jumped from about 10-15Mbps per customer up to 30 to 83Mbps, depending on time of day and overhead. We also upped the channel width from 20MHz to 40MHz so theoretically that alone would account for some improvement. However, these are APs with as many as 63 users on them so really, it’s a pretty impressive jump. Since Galactus isn’t quite ready, we thought of upgrading the Rocket Ms to Rocket AC-Lites but testing showed that particular function isn’t ready for prime time. 802.11ac to 802.11ac radios work fine with 7.14 firmware but the “firmware who must not be named” is still cooking in the pot, meaning it’s simply not ready. We did some testing with it and it looks promising. When it’s finished and you can upgrade a Rocket 5M to a Rocket AC-Lite, it will extend the life of your 802.11n client radios another year by adding 40-80Mbps at the AP side. Just have to hold your breath for a while longer.

In the meantime, Galactus waits for no firmware upgrades. It’s starting to take shape, things are looking very good, and if you aren’t hammering cable early next year, you should get coal in your stocking. I can’t stress this enough, 2016 is the year that wireless technologies need to heavily move into suburbia. Suburbia is where the users per block exceed some of the users per county in rural areas. We are working with one area where the density is about 1600 homes in half a square mile so getting 25% of those potential customers is important. And here is where the strategy changes a little bit.

If you run financial models, the best thing to do is front load as many users as possible during growth periods. Conventional wisdom is to drive the ARPU up. However, it is better to have a lower ARPU up front, even to the point of subsidizing the installation costs, to get as many users online as fast as possible. As in many Roulette strategies though, it also takes the most capital. You can run the numbers yourself but if you have 2000 users paying you $50 a month, that’s $100,000 per month. If you have 4000 users paying you $40 per month, that’s $160,000 per month. Even new math can’t screw that one up considering you are paying somewhere between 50 cents and $4.00 per MB and you only need 2-4Mbps per user if you aren’t controlling your bandwidth. Add a Barracuda box, some fancy Mikrotik programming, or a Procera box and your bandwidth gets even more efficient. There are two points here for high-density deployments and this is the first.

The second lesson here is that the more users you get off cable and DSL in these environments, the more you get to control the spectrum in the area. When your outdoor PTMP system needs 5GHz for maximum speeds, you want your customers on 2.4GHz indoor APs. If you have to deal with cable and DSL providers shipping dual-band radios, then you need to get that customer under your control so you can get them out of 5GHz inside. Since most devices now support auto 20/40MHz channels in 2.4GHz, you can get 150Mbps or more to laptops, phones and tablets without using 5GHz.

I’m telling you that the opportunity to compete with cable has never been better. Plan for it now or miss out while someone else does it for you. So, get the prices down to where you can make a deal no sane person would resist, sell some wireless speeds at value prices, and start getting users off the copper track. It may be Christmas but Santa isn’t the only big guy coming to town, Galactus, Destroyer of Wired Worlds is right behind him.

About Rory Conaway

Rory Conaway has been in the IT and Wireless Industries for the past 25 years as an author and consultant. He currently operates a growing WISP operation in Southern Arizona. He consults with investors, manufacturers, and WISPs, and develops financial business models for startups. In addition to writing articles in industry publications such as Mission Critical Magazine, Mr. Conaway also writes the series “Tales from the Towers” that can be found on various such as www.triadwireless.net and www.muniwireless.com. He has also engineered several wireless designs such as S.P.I.R.I.T. and Guerilla Wireless as well as building integrated wireless and video surveillance for airport security, municipal and critical infrastructure, SCADA systems, and hotel/MDU deployments.

Comments

  1. Mike Hammett says:

    In one area you mention Extreme and another you mention Exalt. I assume you mean Exalt as I don’t think Extreme offers anything in our space?

    In one spot, you mention 20 mbps vs. 40 mbps. I think you mean MHz.

    Also, if you have any serious density of 2.4 for customer internal networks, they will have interference and you’ll need 5 gig for customer internal.

  2. Rory Conaway says:

    Oops, I’ll fix both of those, thanks for the heads up. Too much Christmas eggnog and too little sleep.

    We have very high-density of users but haven’t seen the interference. Keep in mind that if you manage the indoor radio, you get to set the power. Big advantage.

  3. Thanks for sharing the information. As a purveyor of TV White Space systems, who has been struggling to achieve traction in the market, your perspective regarding a desired target 4-5 month ROI is quite valuable. Even in best case scenarios with the promotional opportunities Runcom offers now, ROI for 30 customers is approximately 19 months. $22,885 for full TVWS system with 30 CPEs: $40 per month * 30 users = 19.07 month ROI. Of course the non-line of sight capabilities are wonderful but the market is severely limited by the throughput bottleneck of each Access Point with the current technology
    (December 2015).

  4. Mike Hammett says:

    One way to help that is simple, don’t charge $40. Charge $60. That only brings it down to just under 13 months and not four to five, but its better.

    The only use case I have for it is say a $250 setup and $100/month. Target wealthier clients with it. That brings the equipment cost only down to just over five months.

  5. Rory Conaway says:

    I agree with Mike, because of the cost, you have to target the product in areas with very high ARPU’s. You could also do something like either have the customer cover all the Capex up front or even better, have them lease the equipment as part of the monthly cost. For example, $15 per month to lease the equipment on top the bandwidth income adds an additional $360 over 2 years. We are doing something similar in some areas to bring the install costs down and to make sure we manage the indoor.

    Another option to minimize how many areas you need White Space is to use a NLOS design we came up with a few years ago that seems to work well in some cases with OFDM. Instead of directional antennas, use sectors. Then do a lot of hopping to get to clients. We have as many as 7 hops and they are working great. We have done this for years and shortens your links up significantly.

  6. jjonsson says:

    If using the WS-6-MINI with an AirFiber 5/24 just use the PoE from the AF to power the WS-6-MINI.
    I do that a lot. It waste one port on the switch, but I use that for management when on site….

  7. jjonsson says:

    Also, you can power an EdgeRouter X directly from the WS-6-MINI switch. That gives you 1 extra 24V port to use with PtP as backup link….

  8. Rory Conaway says:

    We get bigger power supplies for the WS-6 switches since we have AF24’s on them. The Edgerouter can’t handle Arizona heat or we would be using them. That’s why we are using Mikrotiks in the NEMA boxes.

  9. Rory, I respect your views and like your posts, but I disagree with the notion that we need to tackle urban areas with unlicensed or even semi licensed technology (1600 per half square mile in your example.)

    Like you, I have done this long enough, and have seen folks like us capture underserved urban areas, only to see the rug pulled under their feet when the cable operator rolls in with a 100mbit $90/month plans (where no smart Mikrotik queues needed as the movie is done downloading before you hit play…) And no, not even a micro cell of cute M&M AC units linked to a nearby 40mhz AC sector, can fend off the wired guys. Heck, can’t even fend off the LTE folks, now providing real throughout of north of 40mbit and ever increasing data caps that can allow you to watch Lost nightly, 25 days without going over your limit… why waste the finite resource at the towers on areas where we can’t add value let alone be appreciated?

    Instead, I think WISPs should focus on markets where the technology shines, where fiber will never go, and where what we do is appreciated without having to erode arpu. Rural ranches, rural business, or as I like to put it, “where you can’t see your neighbors…”

    Cheers,
    Ofer

  10. Mike Hammett says:

    I too think there is plenty of suburban and urban potential.

  11. Rory Conaway says:

    We have started deploying Galactus and currently have 15 users on it. Average speeds are around 150Mbps with peak speeds of about 190Mbps. Since this area is only has 200Mbps maximum speeds, that’s pretty solid. We are upgrading the area right now to 400Mbps to see how far Galactus to go.

  12. You speak a lot about your backhauls with this, but what kind of PTMP gear and antennas are you using? Are you able to get away with this using an omni? I’m just having trouble imagining putting sectors on top of someone’s house.

  13. Rory Conaway says:

    We started with Ubiquiti M series and Omni’s. They work okay, we have hundreds of people on them, but they don’t handle vegetation well. Galactus is next generation. I can talk about results but can’t release details until we get past testing. All I can tell you is stay tuned.

  14. I’m fascinated by the fact that your analysis of the situation is the same as mine. I’ve worked for a couple of the big telecoms on the Enterprise side for the last 10 years and I’ve been stuffing paychecks in my mattress waiting for this round of equipment. Now it’s here and I’m getting things rolling. My area is ideal for this. I live in a town of roughly 8000 where Frontier has DSL dirt cheap but lousy. Then Time Warner is here. The max you can get from them is 50×5 for $89 per month but that’s supposed to jump to 300×20 by the end of the year. But they stop at the end of the city proper. There are 4000 potential customers within a 6 mile radius. Half of them is $200,000 per month at the prices they’re currently paying for HughesNet. And they’re paying for Dish because they can’t stream.
    I have some test customers up running 100Mx20 and a single UBNT Rocket AC with a 45 degree sector antenna out to 4 miles. When I roll the products, everyone is getting a Roku just for signing up. I managed to get Netflix, Amazon, and Hulu to offer some special deals. Cord cutting is complex because everyone wants something different, but here’s a really neat site to direct people to. http://www.theverge.com/a/online-tv-stream-price-guide
    So there’s 2 parts of your triple play. If you really want to deal with part 3, just partner with a 3rd party VoIP provider where you can make residual income but not have to deal with servicing the product. I’m actually talking with a 3rd Party VoIP provider about being able to create a backhaul link straight into one of their pops about 200 miles away. In that case, I could put QOS on the network, and drop it straight onto their network where they then connect to the PSTN and offer business customers SIP trunks without putting their voice over the public internet which would keep the voice quality nice and consistent. 🙂
    Anyways, just some thoughts there on handling the “triple play”. You don’t have to offer each product on your own. Network with partners to come up with the most efficient and cost-effective means of WOWing the customers and knock these cable operators on their rear. You just have to be smart about where you put your stuff and be aware of how you fit into the picture. My method of doing that was to have my daughter go door to door doing a survey on bandwidth and customer satisfaction. I did that 4 months ago so I have a nice database already of unhappy customers and I know what they’re paying. And if I pick up 3/4 of them I’m making a killing. 🙂
    I’m gonna take this whole city over with my own version of GALACTAGUS and when I’m halfway there, I’m moving to the next towns that don’t even have cable. And it feels terrific to see that someone with so much more experience than I have in this came up with the same analysis that I did. GERONIMO!!!!!

  15. Rory … I for sure know what you are saying … our deployed model is something like that … we are only going to 20 MHz/AP for now because we don’t need more vs what competition is offering. We really bet on the new tech and is working for us. I’m pretty sure your Galactus is not close to us … but we are probably not that far away either 🙂

    A good day to you sir!

  16. Hey Rory.. Been a while whens your next article due? Great articles man the whole Tales from the towers series got hooked read one go to the other!
    Galactus is next generation!! Cant wait for your say/testing/feedback a lot of all-round knowledge picked up from these articles
    Keep up the good work but lay off the politics a little its all headache! haha