Provisioning Server Enhancement #1
Patented improvements for enterprise and public DHCP servers

By: Bruce Bahlmann - Contributing Author (your feedback is important to us!)

Created: May 1, 1998

Note: For help designing your provisioning system or developing tools to help test, automate, and deploy your system contact Birds-Eye.Net.

Overview:

Existing IP address provisioning DHCP* servers geared for enterprise networking currently offer only minimum functionality need by operators of a Multiple Subscriber Organizations (MSO) cable modem systems. The following is a list of feature enhancements that could be made by DHCP vendors to empower MSOs to deliver advanced services and increase operational efficiencies:

1. Augment assignment of IP address ranges to support individually managed primary and secondary networks.
2. Registration of ethernet clients such that they can be associated with a tiered group**
3. Allow an additional customer information field to be added to the registration information for correlation
4. Modular DHCP server with ODBC interface.
5. Provide a means to make renumbering seamless.
6. Redundant DHCP server

**Note that class and group can be used interchangeably, group will be used here in.

Each of these enhancements will be discussed in detail. The object of this document is to describe the business need as well as the possible results of adding this functionality that an MSO needs to diversify its one-size fits all product.

Discuss Individual Enhancements:

Assignment of IP Addresses:

Each MSO is allocated a less than generous block of IP addresses that it must use to the nth degree of efficiency before it is granted additional address blocks. To help conserve IP addresses, non-routable networks (such as Net 10) are frequently used for point-to-point links, non-essential equipment, and cable modems. However, the assignment of these non-routable addresses is greatly simplified if DHCP is used to control these assignments. To perform this function a provisioning server must be able to logically separate IP address ranges that are associated with one another (primary-secondaries) to enable ethernet clients to be associated with a particular subnet or a subnet group.

There are currently two commonly used way to assign IP addresses using DHCP (the third way is what this document is proposing as an enhancement).

1. Static
2. Subnet-less specific
3. Subnet-group-less specific

To enable a ethernet client to be associated with a subnet one should avoid a direct assignment as shown in figure 1.0. This particular assignment is a static assignment or reservation and is used to permanently link a particular ethernet client to a specific IP address or reserve a specific IP address for another purpose (reserved for assignment of a router interface). In any such network creation process it is desirable to immediately reserve any automatic IP address assignments such as the router as mention (this may be in fact an additional enhancement). Static IP address assignment actually brought about the need for Dynamic Host Configuration Protocol (DHCP) which essentially provided an automatic means to distribute and manage IP address information. Its important to note that static assignment can take place at the server, at the client, or both. Obviously the least desirable is static assignment involving the client.

Figure 1.0 Static IP Address Assignment

Figure 1.1 describes how a majority of the more sophisticated DHCP servers are set up to handle multiple subnets. In a subnet-less specific model the relay agent (router) provides a logical link to which IP address range the DHCP server should use to dispense IP addresses. This logical link is configured in the DHCP server such that the router’s primary IP address is used to identify which network (and associated available IP address pools) the server should use to satisfy valid requests. Valid requests are those that come from registered clients as un-registered clients will be ignored OR all clients registered and un-registered are treated alike. 

Figure 1.1 Subnet-less Specific Assignment

However this model is limited due to the fact that if secondary address ranges are non-routable and are ear-marked for specific devices, the subnet-less assignment model will treat all related address blocks (secondaries) as one contiguous block. In this case, the only way to specify a client to particular address range is to build additional DHCP servers with unique non-routable address blocks configured and register the ear-marked clients to the sever that manages the desired address range. This solution is costly as hardware must be added for each additional service variation and back systems must be more intelligent to register clients to the server associated with the desired service class. To alleviate this problem, another layer of logic is proposed that would allow a single server to hand out correct IP address ranges based on address type. The desired address types are as follows:

1. Default address type – essentially allows the subnet-less specific model above to work without modification.
2. User defined address type – enables the user to associate an arbitrary <name> with a particular primary OR secondary IP address range.

If a primary OR secondary IP address range has a address type associated with it, that particular IP address range can only be dispensed to a registered client that is associated with that address type. Subnet-less specific model is critical not only for maintaining existing uses of DHCP but also to support auto-provisioning (self-provisioning). In the self-provisioning model a client that is not registered must be allowed to receive an IP address. This is how normal DHCP servers work and self-provisioning can more easily leverage normal DHCP operation (with registration turned off) than build this as add-on functionality. Simply using a default (name-less) IP address type allows unregistered clients to receive a IP address dispensed from a desired IP address range without any DHCP server customization. The only required customization needed is associating a user defined address type to each IP address range that is to be managed (used by registered clients).

Figure 1.2 below describes the process of assigning IP addresses using Subnet-group-less assignment. A request is forwarded to the server via a relay agent and the IP address dispensed to the client will depend on whether the client is registered (or not). Registered clients will receive an address additionally dependent on what address type is associated with them. If no address type is associated with them OR they are unregistered, they will defer to the default address type.

Figure 1.2 Subnet-group-less Assignment

#jdbdump format
<mac>|<device type>|<length>|<ip address>|<lg>|<le>|<lr>|<lt>|<server ip>|<hostname>|<domain>|<address type>

Registration of Ethernet Clients:

Another particularly attractive feature for DHCP servers is to assign individual DHCP options by client or by client group. Some DHCP servers allow this low level of assignment however the means is usually limited either by functionality or scalability. A very powerful way to register a client would be to associate it with a group and allow assignment of various other DHCP options based on their level of specificity. Ideally the client groups should be tiered to enable very particular (base level) DHCP options at the low level (for example bootfile name) and more global DHCP options to be set at the higher (parent) level. In Figure 2.0 the client can be registered to any base group below (basic, sym, best). By registering a client to a base group it automatically obtains the DHCP options set at the parent group. Here, the service group can be defined across many base groups and permits fast and easy administration.

Figure 2.0 Group Associations

The level of specificity is important because a client can pick up DHCP options through various stages of the request processing. Figure 2.1 shows the process a client would go through if this functionality existed in a DHCP server. Note that the options defined initially (server global level) can be overwritten by anything that follows (DHCP options closest to the client have priority over options defined further from the client). However, any options uniquely defined in any phase of the assignment should trickle down the to final response handed to the client.

Figure 2.1 DHCP Options Assignment

#<mac>|<device type>|<length>|<ip address>|<lg>|<le>|<lr>|<lt>|<server ip>|<hostname>|<domain>|<address type>|<group>
#dhcpcap format
#<group name>:<userclass definition>:<dhcp options>:
multiuser_group_2:uc=mu2:bf=multi-user-2:
residential_group:uc=res:

Note that a similar assignment of tiered options can also be useful in subnet base and parent groups. Figure 2.2 shows a typical relationship between the subnet base and parent groups that can be utilized by an MSO. This particular example shows the power group relationship have on local and global levels.

Figure 2.2 Tiered Subnet Group Assignment Example

In this example the group relationships can be used to assign subnet specific DHCP options associated with a headend (HE) and allow parent options to trickle down when desired. A recommended use of this is to create two HE groups and a renumbering group that can be associated with every subnet in a HE. The HE groups would contain DNS servers (a DHCP option) that can be used to load share which DNS server(s) each client uses (depending on which network they reside). The lease time parameters (other DHCP options) should be pushed up to the parent group allowing an administrator to change these times for all subnets managed by a server at once. A special subnet base group called renumber provides the ability to over-ride global lease time parameters for the purpose of subnet renumbering.

Any assignments set up during a registration process must also have the ability to be overwritten. In other words, any existing registration assignment contained in the database can be overwritten without knowledge of its existing settings. The overwrite function, should consist of just invoking a new registration. The new assignments must perform the following functions:

• Overwrite existing information.

• Any previous assignments change to the values contained in the new registration (including blanks).
• Note exception characters (such as pipe ‘|’, equals ‘=’, and colon ‘:’) should be not be part of usable names.

• Cascade registration assignment changes to the client.

• Some mechanism must be in place to trickle down these changes ASAP to client.

• Easiest way to do this would be to expire the clients current lease. In this way, the next renewal/rebind will force client into receiving a new lease that contains the new registration changes.

• Clients that receive a response from the server shortly after the new registration but before the lease is expired should only see changes to their registration group and NOT their subnet group. Subnet groups should only change once the client either expires a lease OR changes their subnet (roams the network).

• Proper handling of BOOTP clients

• Since BOOTP clients typically obtain a relatively infinite lease, changes to their registration information will represent the largest challenge to trickle down. Since BOOTP does not understand a lease or the fact that it needs to periodically check-in with the DHCP server some external stimulation may be needed by a customer to force a cable modem to recycle (in the case of BOOTP).

Customer Information Field:

A customer information field is an additional field that is associated with registered mac addresses. The use of this field is important to the integration of a DHCP server with any other type of database in that it provides a means by which indexes used in the external database can be associated with one or more registered mac addresses in the DHCP server’s database. Without the use of a customer information field, multiple mac addresses that are registered in a DHCP server would need to be looked up individually. However, with the customer information field, a single query of the DHCP server’s database* would result in the listing of all mac addresses associated with a customers primary index.

*Note that this functionality is not typically found in DHCP server’s database, but rather its registration database which is usually some other production database (LDAP, Oracle, etc). The idea of indexing a customer information field to one or more registered mac addresses in the DHCP server is desirable in the absence of a full fledged database.

where <key> can represent either a mac address or a customer info field and result is a listing of all matches.

Modular DHCP server with ODBC Interface:

This biggest problem with DHCP servers today is that while they all conform to a standard protocol for communicating with ethernet clients connected to a network, few (if any) follow any standards as far as interfacing with them. In other words, they all have their own proprietary way of storing lease and client registry data. There are also no standard APIs to configure these servers. The only known exception is Join’s use of a standard bootptab file to set DHCP options for its groups, nodes, and subnets and logic in associating requests from particular relay agents to available IP address pools. However, defining the subnets and dynamic ranges is left to vendor discretion and there is currently no such standard.

A relatively current initiative involving directory enabled networks (DEN) has surfaced (perhaps in response to this problem of interfacing with disparate DHCP servers). The DEN* controlled DHCP server would be directed by some portion of a central database that is used to manage a collection of DHCP servers. Through this database, all servers would receive their configuration information and the configuration database would be administered centrally (perhaps using some type of HPA application – see separate cover on Headend Provisioning Agent for further information on centralized management of multiple DHCP servers). DEN seems to lean in the direction of using LDAP servers as its primary data store for configuration information. This technology, though infant, seems to lend it self well to providing much of this functionality (configuration-wise). However, the use of LDAP for purposes that it is not well designed to handle (like storing fluid or changing information – like lease parameters) may hinder its widespread use with this type of data. LDAP has a very specific use and the need for a muti-threaded relational database (like lease data) has yet to sufficiently scale to an LDAP architecture.

*Note that one important exception that DEN does not address is redundant DHCP services.

Figure 4.0 Modular DHCP Server Model

To deal with this problem, a more usable DHCP service may be to modularize its key components as shown in Figure 4.0. In this means, providing large scale DHCP services is not the responsibility of any one vendor’s product but rather several. Using an extremely lightweight/efficient DHCP engine along with several other required components, the collection of all of them represents a working IP address management system. In this distributed model, the engine would actually provide a type of API or ODBC interface to popular database servers. The recommended database servers would be chosen for their ability to fill the operational requirement defined by the DHCP engine but actually be maintained by an organization skilled at database administration.

Figure 4.1 Master-Slave operation of Modular DHCP Server Model

Because the DHCP server engine is separate from the actual application many different orientations are possible. Figure 4.1 shows one possible orientation that utilizes a master-slave relationship between the Master DHCP engine that could work out of some large data center and a remote or slave DHCP server engine that was located closer to clients. In the master-slave model, the master DHCP server could have read/write access to the DHCP data and the slave DHCP server could have read-only access to DHCP data. This would enable the two to work cooperatively to handle un-expired leases renewals to clients. Since the DHCP data is separate from the master other orientations such as redundant and other variations of the master-slave distributed model are possible. Note that in this modular DHCP world, any company with the know-how to develop a faster, more robust, or fault-tolerant server could enter into this market and satisfy any functional operations required by the IP address management system.

Figure 4.1.1 3^rd Party Applications use of DHCP Data

This modular functionality would also provide endless expansion (see Figure 4.1.1) to the overall model of DHCP services and give rise 3^rd party applications and operations tools that will drive these services to a higher level of operational performance and efficiency. Some examples of these 3^rd party applications would be IP address management, Dynamic DNS, and various troubleshooting or operational efficiency tools. If DHCP data was standardized, tools could simply perform SQL queries for necessary data. Today’s vendor driven model of DHCP servers does not provide much opportunity for 3^rd party applications in this area – mostly due to the fact that each DHCP vendor chooses to store DHCP data on a different type of database. Until these databases are standardized, there will not be much opportunity for expanded functionality in the DHCP server space because the vendor driven model will be less diverse than a modular DHCP model that would be driven by 3^rd party innovation.

Failing to breakup the DHCP server model will cause each vendor to continue to go their own way and implement only the DHCP services that are in that vendor’s (or what each vendor views as their customers’) best interest. Going down this path will mean that providing DHCP services requires using a single vendor solution. The complications of this model operations-wise is endless and the business model is disastrous. Businesses geared for using a particular DHCP server could not easily convert to less costly solution due to the non-standard way each and every server works. If a particular server were to be discontinued or significantly changed, the operations of the organization would be ‘forced’ to pay the price.

Moving to a modular DHCP model would enable growth in a company dependent on DHCP services and place each and every vendor on a level playing field as far as performance and service and provide companies the flexibility to select several DHCP vendors to provide these services. It would also provide room for new companies to enter this space and perhaps drive standards for redundant DHCP services (an area of much debate and disagreement – mainly because each vendor has chosen to implement the operation of their DHCP server differently).

DHCP server vendors that move in this modular direction understand the fact that they cannot be all things to all customers. They recognize the enormous impact that their product has in the success of an MSO’s operations and want to select one service and do that very well and have a head start on other vendors in being a major player in the next round of DHCP applications. MSO’s need to understand this and put their money in DHCP vendors that pursue this model. The action of MSO investing in DHCP vendors seeking a breakup of their own monopoly and build more modular DHCP servers will drive further interest in other DHCP vendors to follow suite and also raise interest of venture capitalists to invest in startup companies wanting to contribute new ways to provide the endless array of DHCP services sought by MSOs.

Seamless Renumbering

A problem with DHCP servers that are used by an MSO is that they do not have sufficient available IP address space to operate in a rapidly expanding environment. In this case, an event called renumbering occurs quite often that redistributes IP addresses to a select portion of clients residing in a network that is highly utilized. The renumber candidates undergo a process where by their existing gateway is changed to a newly introduced network. In this new network, their current DHCP lease becomes invalid and they must invoke some surrender (DHCP release) and acquisition (DHCP discover) process to obtain a working lease from the DHCP server. Due to inefficiencies in the renumbering process (mainly the DHCP server), this often causes clients more than a momentary outage. In fact, the worst case outage for a client to recover from the outage caused by a renumbering is the rebind time (or 87% of the lease time – if the rebind time is not explicitly set). During the rebind time, the client performs a broadcast request to the server which is properly handled by the relay agent to finally allow the client to communicate with the DHCP server. Previous renewals (a type of request that is often sent before a rebind), since they are uni-casts to the DHCP server will not reach the DHCP server because the client does not have the correct gateway to reach the DHCP server directly. This delay in client reconfiguration can be significantly reduced if client leases were allowed to align themselves with a set renumbering time to permit a near-seamless (near zero outage time) renumbering event. In the future as the availability of DHCP services is rated by uptime, seamless renumbering will be one mechanism that administrators will need reduce customer downtime.

In seamless renumbering, client leases belonging to a particular subnet are aligned to expire at a particular future date and time (the exact date and time of a planned renumbering). Clients become aligned with this new date and time (lease parameter wise) when they perform normal DHCP renewals. The DHCP server administrator must make sure that every working client (network devices that are always on) have time enough to complete one renewal before the planned renumbering. During this renewal transferal, the new lease times are propagated to the clients which become aligned with the planned renumbering. Any clients that are off between the time that the renumbering lease times are set will naturally perform the needed DHCP DISCOVER they need to operate on the newly renumbered net. An client that enters the network (or turns on) between the time the renumbering lease times are set and the actual renumbering will be assigned a lease that will expire at the desired renumbering time (a calculation of this correct lease time would need to be done at the server). Note that aligning all clients to perform a rebind may provide the equivalent to the expire with less risk.

When the actual renumbering time is reached, the administrator will simply move the clients over to the new network (which is already configured in DHCP) just moments before the renumbering least time expires and update the existing network with default lease parameters. This will force all working clients to DISCOVER their lease and thus obtain a new working lease.

Can Birds-Eye.Net help you or your Company?
Receive your Birds-Eye.Net articles and white papers hot off the presses by adding our RSS feed to your reader.

(C) Copyright Birds-Eye.Net, All rights reserved.
It is against the law to reproduce this content or any portion of it in any form without the explicit written permission of Birds-Eye Network Services, LLC. Federal copyright law (17 USC 504) makes it illegal, punishable with fines up to $100,000 per violation plus attorney's fees.