Birds-Eye.Net Design for Autoprovisioning and Multi-User Support

Multi-User & Auto-Provisioning Support
Soup to nuts technical overview of how to implement auto-provisioning

By: Bruce Bahlmann - Contributing Author (your feedback is important to us!)

Discuss systems configuration and interfaces necessary to support autoprovisioning and multi-user functionality with regard to high speed Internet services.

Headend Provisioning Agent (hpa.pl) version 3.0 or higher (configures DHCP server with multiple service groups)

Figure 1.0 introduces the major components of the provisioning system. The most important component being the integration of troubleshooting & management tools and the self service interface. While this system ushers in the self service interface, ETE must continue to be supported for managing customer cable modems and personal computers until such time as the self service interface can support installers, Tier1 & Tier2, and customers. Until such time as the self service interface manages this space, all additional functionality (multi-user support and autoprovisioning) is dependent on changes in ETE to enable new products and services.

OSS enhancements center mainly around modifications to the scripts that glue various systems together. The two scripts involved in this enhancement would be the provisioning script (located on the ETE server), and pserver which is located on the DHCP server. The normal interaction between these scripts is the following:

The proposed changes to the provisioning and pserver script would enable multi-user functionality as well as the ability to offer "new" services at lower or higher bandwidth. Note that in the above example, the "number of pcs" would be appended to the service group only if the service group name was "mu" (i.e. mu2). In this way, we could also support customers who only want to pay for one Internet connection but actually have two computers.

ETE enhancements center around providing the initial datastore for DHCP server provisioning data to eventually providing a "view" of the data which is moved to LDAP. ETE must support the following requirements to enable multi-user functionality:

Several configurations must be properly set up in the DHCP/BOOTP/TFTP server to enable multi-user and autoprovisioning services. These configurations are grouped into four categories: DHCPCAP Service Group Structure, Subnet Allocation, DHCPCAP Network Structure, and MD5 File Structure. Each of these configurations are discussed below.

The DHCPCAP Service Group Structure is "new" functionality that enables Join to support multiple classes of service (or more appropriately service groups). The service group feature is what enables multi-user capability along with allowing an MSO to further diversify its Internet service product line (lower or higher access service groups). In the proposed two server model this DHCPCAP configuration is sent to BOOTP/.DHCP server to enable legacy (LANCity) and DOCSIS support for cable modems. The contents of this configuration line means the following:

Field:	Name:	Example:
<name>	Reference name	:DOCSIS_basic:
<uc>	User-defined class	:uc=basic:
<hd>	Home directory	:hd=/DOCSIS:

Field:	Name:	Example:
<name>	Reference name	:LANCity_symmetric:
<uc>	User-defined class	:uc=symmetric:
<bf>	Bootfile	:bf=symmetric:

The reason that the two service groups are different has to do with the fact that legacy cable modems require network specific parameters in the bootfile (namely their default gateway but also forward/return frequencies which can vary from headend). While DOCSIS cable modems can also have forward/return frequencies in their bootfile they are not required and for simplicity sake should not be used. The more generic one can make the bootfile responsible for a service group the easier that service group will be to maintain. The different service group definitions enable a single network definition to support both legacy and DOCSIS cable modems. This is an important feature as the availability of IP addresses is less than plentiful and efficient use of address space is strongly advised. Without this method of making more efficient use of IP address space, DOCSIS cable modems would require their own subnet causing twice as many IP addresses to be tied up for the same number of clients.

The service group configuration works closely with the DHCPCAP network settings that will be discussed later. Before a network is created it’s a good idea to understand how subnets should be allocated to satisfy the needs of autoprovisioning. To perform autoprovisioning one should have at least 2 subnets (three is highly recommended). The following is the recommended allocation of a two subnet model:

Network:	Netmask Bits:	Server:	From:	To:	Use:
209.32.160.0	/23	DHCP	209.32.160.2	209.32.161.254	Customer PCs
10.32.160.0	/23	BOOTP/DHCP	10.32.160.2	10.32.160.10	Headend Nodes
			10.32.160.11	10.32.161.200	Customer CMs
			10.32.161.201	10.32.161.254	25 DOCSIS AP customers

In a two subnet allocation model the primary subnet (209.32.160.0) is reserved for customer PCs. The primary address pool is protected by requiring clients to be registered before they can receive an IP address. The secondary for this network (10.32.160.0) must handle all the rest of the clients which need an IP address. Care must be taken to ensure that routable IP addresses only fall into the hands of paying customers rather than network administrative equipment (our ability to do this will distinguish us from our competitors). Unfortunately, the secondary subnet must be sliced and diced to support autoprovisioning. The biggest problem with only using 2 subnets is it that it becomes difficult to manage all these address pools used by the customer PCs, headend nodes (for legacy headend nodes), customer cable modems (CMs), and autoprovisioning customers. This problem occurs because there does not exist a one-to-one relationship between routable IP addresses for customer PCs and non-routable IP addresses for customer CMs. Since a "majority" of our customers will only have a single computer connected to a single cable modem this would require that we should plan for about as many routable IP addresses to be consumed by residential customers as non-routable IP addresses. To support this operationally one should either build tools to continually track available addresses in each pool and send alerts when they reach some defined level (idea! for new product) – OR – use a second secondary subnet to divide PCs and CMs into equal address pools. The following introduces the a second secondary that is used primarily for autoprovisioning and other statically addressed devices (like legacy headend nodes). One should not assume that legacy cable modems will be entirely replaced by DOCSIS technology. Legacy cable modems have an on-going role in MSO network operations whether its Institutional Networks (Inets) or links between local MSO offices – Legacy cable modems are field proven and MSO will use them as long as they work.

Network:	Netmask Bits:	Server:	From:	To:	Use:
209.32.160.0	/23	DHCP	209.32.160.2	209.32.161.254	Customer PCs
10.32.160.0	/23	BOOTP/DHCP	10.32.160.2	10.32.161.254	Customer CMs
10.132.160.0	/23	BOOTP/DHCP	10.132.160.2	10.132.160.11	Headend Nodes
			10.132.160.12	10.132.161.254	DOCSIS AP customers

This model allows the customer CM and PC pools to mirror one another thus making it easier to manage the address space. Autoprovisioning customers will use what is left of the second secondary which is more than enough space than we should need to support these customers as well as cable modems (note that an autoprovisioning customer represents 2 IP addresses – 1 for CM and 1 for PC).

Product Suggestion: While a whole two address block seems like overkill to support autoprovisioning there may be other devices which require an address that can not be mapped into the regular customer CM pool. An example may be the new set top boxes. If set top boxes require an IP address to work we will need to come up with an IP address pool to support these (potentially their own unique bootfile). Another use of the autoprovisioning IP address space could be utilized by a new Internet service group. Customers who sign on to this group would utilize a 10net IP address for both their cable modem and their PC. Such a service would not consume a routable IP address space but take advantage of the available secondary address space and proxy servers already in place the service could be provided at a lower cost than one with a routable IP address (our infrastructure already supports this capability we just need to exploit it).Therefore its important to keep some space available to support ‘future’ demands of the provisioning system.

Configuring the network structure on the DHCP server is one of the most complex components of supporting autoprovisioning. This process must allow existing functionality to work yet enable unregistered clients join the network in a limited state of operation. This limited state is controlled by the cable modem receiving a restricted access configuration and the PC receiving a restricted lease. These two important operations enable a limited access state that would require some type of self service interface for the customer to sign up for service without the MSO’s intervention. The following is the network configuration for the three subnet and two server model.

Roseville_209.32.160.0:nw=209.32.160.0:sm=255.255.254.0:tc=roseville_1:gw=209.32.160.1:

Field:	Name:	Example:
<name>	Reference name	:Roseville_209.32.160.0:
<nw>	Network	:nw=209.32.160.0:
<sm>	Subnet mask	:sm=255.255.254.0:
<tc>	Subnet base group	:tc=roseville_1:
<gw>	Gateway address	:gw=209.32.160.1:

The configuration above will allow a registered PC to get an IP address in the routable IP address range if it resides within the defined network (the gateway address [relay agent] of the requesting client enable the DHCP server to map clients to this configuration and associated IP address pools). The network for the PCs is the easiest to set up because there is no logic needed to support cable modems (bootfile mappings) or autoprovisioning. It is in this way that dividing the tasks of supporting various clients can take advantage of a two server model. In the two server model, the DHCP server is only responsible for routable IP addresses so it can be optimized to perform just one task: provide DHCP services for registered clients. When this one task is complicated with all the logic needed to support various other services (like BOOTP, autoprovisioning, etc) important tasks like responding to active clients may not be performed optimally. The BOOTP/DHCP server in the two server model handles the remainder of tasks necessary to support legacy & DOCSIS cable modems as well as provide autoprovisioning functionality. By supporting all these different devices and services, the BOOTP/DHCP server configuration is somewhat more complex than that of the DHCP server and by comparison, the BOOTP/DHCP server is sufficiently more burdened than the DHCP server. However, supporting cable modems is generally less work for a BOOTP/DHCP server so this evens out the workload better. The following is a how the first secondary network would be set up on the BOOTP/DHCP server.

Roseville_10.32.160.0:nw=10.32.160.0:sm=255.255.254.0:tc=roseville_1:gw=10.32.160.1:bf=basic:hd=/roseville/209.32.160.0:

Field:	Name:	Example:
<name>	Reference name	:Roseville_10.32.160.0:
<nw>	Network	:nw=10.32.160.0:
<sm>	Subnet mask	:sm=255.255.254.0:
<tc>	Subnet base group	:tc=roseville_1:
<gw>	Gateway address	:gw=10.32.160.1:
<bf>	Subnet default bootfile	:bf=basic:
<hd>	Subnet home directory	:hd=/roseville/209.32.160.0:

In this configuration some additional parameters are added to the network configuration to support "registered" cable modem access to the BOOTP/DHCP server (note that the primary network address is always used to identify the network – that is the reason the 209.32.160.0 is used in the TFTP path name). The actual response that registered clients receive depends on whether additional configurations (DHCP options) are applied. Less specific configuration settings such as those associated with the subnet have lower priority over those defined closer to the client (such as service group DHCP options). For example, the only way the default bootfile set above will make it to the client is if this setting is not overwritten with something else or cleared (:bf=:) somewhere down the line. The "cm" indicator in the NETS file allows the DHCP server to assign registered hosts to this subnet group (this is a completely different group from the service group).

Possible DHCP Server Enhancement: Further provisioning server operations could be enhanced IF there were a separate home directory (hd) parameter for DHCP and BOOTP. Meaning that if the server was answering a DHCP or BOOTP request it would use the default home directory parameter (for backward compatibility) unless an additional home directory parameter was present (i.e. DHCP home directory). In this case, the DHCP response would contain the special DHCP home directory and the BOOTP response would contain the BOOTP home directory (the default one). This enhancement would also simplify the service groups defined above allowing DOCSIS and legacy cable modems to share the same service groups. In other words, each new service group would only mean one additional configuration because the separate DHCP and BOOTP home directories would direct both cable modem types to their correct bootfile locations. It would also avoid having to copy the same unregistered bootfile into all second secondary TFTP directory structures (only way to have a single network support both legacy headend nodes and DOCSIS autoprovisioning). Note that without the second home directory configuration, MSO would only be able to perform autoprovisioning with one type of modem. However, with this configuration, both types of modems would be able to perform autprovisioning.

Roseville_10.132.160.0:nw=10.132.160.0:sm=255.255.254.0:tc=limited:gw=10.32.160.1:bf=unreg:hd=/roseville/209.32.160.0/10.132.160.0:

Field:	Name:	Example:
<name>	Reference name	:Roseville_10.132.160.0:
<nw>	Network	:nw=10.132.160.0:
<sm>	Subnet mask	:sm=255.255.254.0:
<tc>	Subnet base group	:tc=limited:
<gw>	Gateway address	:gw=10.132.160.1:
<bf>	Subnet default bootfile	:bf=unreg:
<hd>	Subnet home directory	:hd=/roseville/209.32.160.0/10.132.160.0:

In the second secondary network configuration there are some differences from the previous networks. In this configuration, the subnet base group (tc) associated with this network is in a limited class that would prevent unauthorized Internet access. The bootfile name (bf) is configured as unregistered (unreg) and since the subnet group is <blank> for this subnet this information will be given out to any client that requests that is NOT registered.

Until some type of self service site is up and running the unreg bootfile may just be a boot file that has network access shut off. This allows an MSO to quiet unregistered cable modems by allowing them to boot successfully but does not give them any network access.

The last component of the provisioning system is to match the home directories and bootfile names configured in the subnets and service groups described previously. This can be accomplished by placing files in the proper directories off the TFTP home directory. Since most TFTP servers use the –s option to allow more secure transactions (prevents clients from downloading anything other than files contained the TFTP home directory and higher), all the home directory configurations will be absolute from the TFTP home directory (so if the TFTP home directory is at /tftpboot and the TFTP demon is running in the secure mode, none of the home directory configurations will contain the /tftpboot since it can be assumed when the client does a TFTP).

* Note that the file named "unreg" placed in this directory can NOT be utilized until the suggested server enhancement is complete. If such functionality existed, the "unreg" bootfile would still exist in the legacy directory structure but rather than supporting only DOCSIS this file then ONLY support legacy cable modems. The result of this would mean that both legacy and DOCSIS cable modems could perform autoprovisioning.

All the operations described above must be carried out with meticulous accuracy for everything to work.