I built my first “cluster” of single-board computers almost immediately after the Orange Pi PC microcomputer began to gain popularity. It could be called a “cluster” with a big stretch, because from a formal point of view it was just a local network of four boards that “saw” each other and could access the Internet.
The device participated in the SETI@home project and even managed to count something. But, unfortunately, no one came to pick me up from this planet.
However, during all this time fiddling with wires, connectors and microSD cards, I learned a lot. So, for example, I found out that you shouldn’t trust the declared power of the power supply, that it would be nice to distribute the load in terms of consumption, and the cross-section of the wire matters.
And yes, we had to “collectively farm” the power management system, because the simultaneous start of five single-board devices may require a starting current of the order of 8-10A (5*2)! This is a lot, especially for power supplies made in the basements of the country, where we love to order all sorts of... interesting gadgets.
I'll probably start with her. The problem came down to relatively simple actions- after a given period of time, sequentially turn on 4 channels through which 5 volts are supplied. The easiest way to implement your plan is Arduino (of which every self-respecting geek has an abundance) and this miracle board from Ali with 4 relays.
And you know, it even worked.
However, the “refrigerator-style” clicks at startup caused some discomfort. Firstly, when there was a click, interference ran through the power supply and it was necessary to install capacitors, and secondly, the entire structure was quite large.
So one day I simply replaced the relay block with transistor switches based on IRL520.
This solved the issue with interference, but since the mosfet controls the “zero”, I had to abandon the brass legs in the rack so as not to accidentally connect the ground of the boards.
And now, the solution is being replicated perfectly and two clusters are already working stably without any surprises. Just as planned.
But let's return to replication. Why buy power supplies for a significant amount of money when there are many affordable ATXs literally under your feet?
Moreover, they have all the voltages (5,12,3.3), the beginnings of self-diagnosis and the possibility of program control.
Well, I won’t go into too much detail here - an article about controlling ATX via Arduino.
Well, are all the pills eaten, and the stamps are also stuck on? It's time to put it all together.
There will be one head node that connects to outside world via WiFi and transfers “internet” to the cluster. It will be powered by ATX standby voltage.
In fact, TBNG is responsible for distributing the Internet.
So, if desired, cluster nodes can be hidden behind TOR.
Also, there will be a tricky board connected via i2c to this head node. It will be able to turn on/off each of the 10 worker nodes. Plus, it will be able to control three 12V fans to cool the entire system.
The operating scenario is as follows: when ATX is turned on at 220V, the head node starts. When the system is ready for operation, it sequentially turns on all 10 nodes and fans.
When the switching process is completed, the head node will go around each working node and ask how we feel, what is the temperature. If one of the racks gets hot, increase the airflow.
Well, with a shutdown command, each of the nodes will be carefully turned off and de-energized.
I drew the board diagram myself, so it looks creepy. However, a well-trained person took on the tracing and manufacturing, for which we thank him very much.
Here it is in the process of being assembled
Here is one of the first sketches of the location of the cluster components. Made on a checkered piece of paper and immortalized via Office Lens by the phone.
The entire structure is placed on a sheet of textolite purchased for the occasion.
This is approximately what the location of the nodes inside looks like. Two racks of five cards each.
Here you can see the Arduino control. It is connected to the head Orange Pi Pc via i2c via a level converter.
Well, here is the final (current version).
So, all you need is to write several utilities in Python that would conduct all this music - turn it on, turn it on, and regulate the fan speed.
I won’t bore you with technical details - it looks something like this:
| 1
2 3 4 5 6 7 8 | #!/usr/bin/env sh echo "Starting ATX board..." /home/zno/i2creobus/i2catx_tool.py --start echo "Setting initial fan values..." /home/zno/i2creobus/i2creobus_tool.py --fan 0 --set 60 /home/zno/i2creobus/i2creobus_tool.py --fan 1 --set 60 /home/zno/i2creobus/i2creobus_tool.py --fan 2 --set 60 |
Since we already have as many as 10 nodes, we use Ansible, which will help, for example, to correctly turn off all nodes. Or run a temperature monitor on each.
| 1
2 3 4 5 6 7 8 | ---
- hosts: workers roles: - webmon_stop - webmon_remove - webmon_install - webmon_start |
I am often accused in a dismissive tone, saying that this is just a local single-plate network (as I already mentioned at the very beginning). In general, I don’t give a damn about other people’s opinions, but perhaps we’ll add some glamor and organize a docker swarm cluster.
The task is very simple and can be completed in less than 10 minutes. Then we launch an instance of Portainer on the head node, and voila!
Now you can really scale tasks. So, in this moment The cluster runs the Verium Reserve cryptocurrency miner. And quite successfully. I hope the nearest native will recoup the consumed electricity;) Well, or reduce the number of nodes involved and mine something else like Turtle Coin.
If you want a payload, you can throw Hadoop into the cluster or arrange balancing of web servers. There are a lot of ready-made images on the Internet, and there is enough training material. Well, if the image (docker image) is missing, you can always build your own.
What did this teach me? In general, the technology “stack” is very broad. Judge for yourself - Docker, Ansible, Python, upgrading Arduino (God forgive me, it won’t be said by night), and the shell, of course. And also KiCad and working with a contractor :).
What can be done better? Much. On the software side, it would be nice to rewrite control utilities in Go. By the way - make it more steampunkish - KDPV at the beginning perfectly raises the bar. So there is something to work on.
Roles performed by:
- Head node - Orange Pi PC with usb wifi.
- Working nodes - Orange Pi PC2 x 10.
- Network - 100 Mbit TP-link@16ports.
- Brain - Arduino clone based on Atmega8 + level converter.
- Heart - controller ATX power supply with power supply.
- Software (soul) - Docker, Ansible, Python 3, a little shell and a little bit of laziness.
- The time spent is priceless.
During the experiments, a couple of Orange Pi PC2 boards were damaged due to a mixed-up power supply (they burn very beautifully), another PC2 lost Ethernet (this is a separate story in which I do not understand the physics of the process).
That seems to be the whole story “from top to bottom.” If anyone finds it interesting, ask questions in the comments. And vote for questions there (upvote - each comment has a button for this). The most interesting questions will be covered in new notes.
Thank you for reading to the end.
First of all, decide what components and resources you will need. You will need one main node, at least a dozen identical compute nodes, an Ethernet switch, a power distribution unit and a rack. Determine the wiring and cooling capacity, as well as the amount of space you will need. Also decide what IP addresses you want to use for the nodes, what software you will install, and what technologies you will need to create parallel computing power (more on this below).
- Although hardware is expensive, all the programs presented in the article are distributed free of charge, and most of them are open source.
- If you want to find out how fast your supercomputer could theoretically be, use this tool:
Mount the nodes. You will need to assemble network nodes or purchase pre-assembled servers.
- Choose server frames that maximize space, energy efficiency, and cooling performance.
- Or you can “recycle” a dozen or so used servers, a few outdated ones - and even if their weight exceeds the total weight of the components, but you will save a decent amount. All processors, network adapters and motherboards must be the same for the computers to work well together. Of course, don't forget about RAM and hard disks for each node, as well as at least one optical drive for the main node.
Install the servers in the rack. Start at the bottom so that the rack is not overloaded at the top. You'll need a friend's help - assembled servers can be very heavy, and placing them in the cages that hold them in the rack is quite a challenge.
Install an Ethernet switch next to the rack. It's worth configuring the switch right away: set the size of jumbo frames to 9000 bytes, set the static IP address that you chose in step 1 and turn off unnecessary protocols such as SMTP.
Install a power distribution unit (PDU, or Power Distribution Unit). Depending on the maximum load the nodes on your network can handle, you may need 220 volts for your high-performance computer.
When everything is installed, proceed to configuration. Linux is, in fact, the go-to system for high-performance (HPC) clusters - not only is it ideal as a scientific computing environment, but you also don't have to pay to install the system on hundreds or even thousands of nodes. Imagine how much it would cost to install Windows on all the nodes!
- Start by installing the latest BIOS version For motherboard and software from the manufacturer, which must be the same for all servers.
- Set your preferred Linux distribution to all nodes, and to the main node - a distribution kit with graphical interface. Popular systems: CentOS, OpenSuse, Scientific Linux, RedHat and SLES.
- The author highly recommends using Rocks Cluster Distribution. In addition to installing all the software and tools needed for a cluster, Rocks provides an excellent method for quickly "migrating" multiple copies of a system to similar servers using PXE boot and Red Hat's "Kick Start" procedure.
Install the message passing interface, resource manager and other required libraries. If you did not install Rocks in the previous step, you will have to manually install the necessary software to configure the parallel computing logic.
- To get started you will need portable system for working with bash, for example, Torque Resource Manager, which allows you to split and distribute tasks across multiple machines.
- Add Maui Cluster Scheduler to Torque to complete the installation.
- Next you need to install a message passing interface, which is necessary to ensure that individual processes in each individual node use the common data. OpenMP is the simplest option.
- Don't forget about multi-threaded math libraries and compilers that will “assemble” your programs for distributed computing. Did I mention you should just play Rocks?
Connect computers into a network. The main node sends tasks for calculation to slave nodes, which in turn must return the result back, and also send messages to each other. And the faster all this happens, the better.
- Use private Ethernet network to connect all nodes into a cluster.
- The master node can also act as an NFS, PXE, DHCP, TFTP and NTP server when connected to Ethernet.
- You must separate this network from the public ones to ensure that packets do not overlap with others on the LAN.
Test the cluster. The last thing you should do before giving users access to computer power is to test the performance. HPL (High Performance Lynpack) benchmark is a popular option for measuring computing speed in a cluster. You need to compile the software from source to the highest degree of optimization that your compiler allows for the architecture you choose.
- You should, of course, compile with all possible optimization settings that are available for the platform you choose. For example, when using AMD CPU compile to Open64 and optimization level -0.
- Compare your results with TOP500.org to pit your cluster against the 500 fastest supercomputers in the world!
Introduction
A server cluster is a group of independent servers managed by the Cluster service that work together as a single system. Server clusters are created by combining multiple Windows® 2000 Advanced Server and Windows 2000 Datacenter Server servers together to provide high level availability, scalability and manageability for resources and applications.
The purpose of a server cluster is to ensure continuous user access to applications and resources in cases of hardware or software failures or planned equipment shutdowns. If one of the cluster servers is unavailable due to a failure or stoppage for execution Maintenance, informational resources and applications are redistributed among the remaining available cluster nodes.
For cluster systems, the use of the term " high availability" is preferable to using the term " fault tolerance", since fault tolerance technologies imply a higher level of equipment resistance to external influences and recovery mechanisms. Typically, fault-tolerant servers use high degree hardware redundancy, plus in addition specialized software that allows almost immediate restoration of operation in the event of any single software or hardware failure. These solutions are significantly more expensive compared to the use of cluster technologies, since organizations are forced to overpay for additional hardware, which is idle most of the time and is used only in case of failures. Fault-tolerant servers are used for applications that handle high volumes of high-value transactions in areas such as payment processing centers, ATMs, or stock exchanges.
Although the Cluster service does not guarantee uptime, it does provide a high level of availability sufficient to run most mission-critical applications. The Cluster service can monitor the performance of applications and resources, automatically recognizing failure conditions and restoring the system when they are resolved. This provides more flexible workload management within the cluster and increases the availability of the system as a whole.
The main benefits obtained by using the Cluster service are:
- High availability. If a node fails, the Cluster service transfers control of resources, such as hard drives and network addresses, to the functioning cluster node. When a software or hardware failure occurs, the cluster software restarts the failed application on the functioning node, or moves the entire load of the failed node to the remaining functioning nodes. However, users may only notice a short delay in service.
- Refund after refusal. The Cluster service automatically redistributes the workload in the cluster when a failed node becomes available again.
- Controllability. Cluster Administrator is a snap-in that you can use to manage your cluster as unified system, as well as for application management. The Cluster Administrator provides a transparent view of how applications are running as if they were running on the same server. You can move applications to different servers within a cluster by dragging and dropping cluster objects with the mouse. You can move data in the same way. This method can be used to manually distribute the workload of servers, as well as to offload the server and then stop it for scheduled maintenance. In addition, the Cluster Administrator allows you to remotely monitor the state of the cluster, all its nodes and resources.
- Scalability. To ensure that cluster performance can always keep up with increasing demands, the Cluster service has scaling capabilities. If the overall cluster performance becomes insufficient to handle the load generated by clustered applications, additional nodes can be added to the cluster.
This document contains instructions for installing the Cluster service on servers running Windows control 2000 Advanced Server and Windows 2000 Datacenter Server, and describes the process of installing the Cluster service on cluster node servers. This guide does not describe installing and configuring clustered applications, but only walks you through the entire process of installing a simple two-node cluster.
System requirements for creating a server cluster
The following checklists will help you prepare for installation. Step-by-step installation instructions will follow these listings.
Software requirements
- operating system Microsoft Windows 2000 Advanced Server or Windows 2000 Datacenter Server installed on all servers in the cluster.
- Installed service name resolution such as Domain Naming System (DNS), Windows Internet Naming System (WINS), HOSTS, etc.
- Terminal server for remote administration cluster. This requirement is not mandatory, but is recommended only to ensure ease of cluster management.
Hardware Requirements
- The cluster node hardware requirements are the same as those for installing the Windows 2000 Advanced Server or Windows 2000 Datacenter Server operating systems. These requirements can be found on the search page Microsoft directory.
- The cluster hardware must be certified and listed on the Microsoft Hardware Compatibility List (HCL) for the Cluster service. Latest version this list can be found on the search page Windows 2000 Hardware Compatibility List Microsoft directory by selecting the search category "Cluster".
Two HCL-compliant computers, each with:
- HDD with a bootable system partition and the Windows 2000 Advanced Server or Windows 2000 Datacenter Server operating system installed. This drive should not be connected to the shared storage bus, discussed below.
- Separate PCI Fiber Channel or SCSI device controller for connecting an external shared storage device. This controller must be present in addition to the controller boot disk.
- Two network PCI adapter installed on each computer in the cluster.
- An HCL-listed external disk storage device that is connected to all nodes in the cluster. It will act as a cluster disk. Configuration using hardware RAID arrays is recommended.
- Cables for connecting a common storage device to all computers. Refer to the manufacturer's documentation for instructions on configuring storage devices. If the connection is made to the SCSI bus, you can refer to Appendix A for additional information.
- All equipment on the cluster computers must be completely identical. This will simplify the configuration process and eliminate potential compatibility issues.
Requirements for setting up network configuration
- A unique NetBIOS name for the cluster.
- Five unique static IP addresses: two addresses for network adapters private network, two for public network adapters, and one address for the cluster.
- Domain account for the Cluster service (all cluster nodes must be members of the same domain)
- Each node must have two network adapters - one for connecting to the public network, one for intra-cluster communication of nodes. Configuration using a single network adapter to connect to both a public and private network simultaneously is not supported. Having a separate network adapter for the private network is required to comply with HCL requirements.
Shared Storage Disk Requirements
- All shared storage disks, including the quorum disk, must be physically attached to the shared bus.
- All disks connected to the shared bus must be accessible by each node. This can be checked during the installation and configuration of the host adapter. For detailed instructions, refer to the adapter manufacturer's documentation.
- SCSI devices must be assigned target unique SCSI ID numbers, and terminators must be installed correctly on the SCSI bus, according to the manufacturer's instructions. 1
- All shared storage drives must be configured as basic disks(not dynamic)
- All shared storage drive partitions must be formatted with the NTFS file system.
It is highly recommended to combine all shared storage drives into hardware RAID arrays. Although not required, creating fault-tolerant RAID configurations is key to protecting against disk failures.
Cluster installation
General Installation Overview
During the installation process, some nodes will be shut down and some will be rebooted. This is necessary in order to ensure the integrity of the data located on disks connected to the common bus of the external storage device. Data corruption can occur when multiple nodes simultaneously attempt to write to the same disk that is not protected by the cluster software.
Table 1 will help you determine which nodes and storage devices should be enabled at each stage of installation.
This guide describes how to create a two-node cluster. However, if you are setting up a cluster with more than two nodes, you can use the column value "Node 2" to determine the state of the remaining nodes.
Table 1. Sequence of turning on devices when installing a cluster
| Step | Node 1 | Node 2 | Storage device | A comment |
| Setting Network Settings | On | On | Off | Make sure that all storage devices connected to the common bus are turned off. Turn on all nodes. |
| Setting up shared drives | On | Off | On | Turn off all nodes. Power on the shared storage device, then power on the first node. |
| Checking the configuration of shared drives | Off | On | On | Turn off the first node, turn on the second node. Repeat for nodes 3 and 4 if necessary. |
| Configuring the first node | On | Off | On | Turn off all nodes; turn on the first node. |
| Configuring the second node | On | On | On | After successfully configuring the first node, power on the second node. Repeat for nodes 3 and 4 if necessary. |
| Completing the installation | On | On | On | At this point, all nodes should be turned on. |
Before installing the cluster software, you must complete next steps:
- Install an operating system on each computer in the cluster Windows system 2000 Advanced Server or Windows 2000 Datacenter Server.
- Configure network settings.
- Configure shared storage drives.
Complete these steps on each node in the cluster before installing the Cluster service on the first node.
To configure the Cluster service on a Windows 2000 server, your account must have administrator rights on each node. All cluster nodes must be either member servers or controllers of the same domain. Mixed use member servers and domain controllers in a cluster are not allowed.
Installing the Windows 2000 operating system
For Windows installations 2000 on each cluster node, refer to the documentation that you received with your operating system.
This document uses the naming structure from the manual "Step-by-Step Guide to a Common Infrastructure for Windows 2000 Server Deployment". However, you can use any names.
Before you begin installing the Cluster service, you must log in as an administrator.
Configuring network settings
Note: At this point in the installation, turn off all shared storage devices, and then turn on all nodes. You must prevent multiple nodes from accessing a shared storage device at the same time until the Cluster service is installed on at least one node and that node is powered on.
Each node must have at least two network adapters installed - one to connect to the public network, and one to connect to the private network consisting of cluster nodes.
The private network adapter provides inter-node communication, reporting of the current state of the cluster, and management of the cluster. Each node's public network adapter connects the cluster to the public network consisting of client computers.
Make sure that all network adapters are physically connected correctly: private network adapters are connected only to other private network adapters, and public network adapters are connected to public network switches. The connection diagram is shown in Figure 1. Perform this test on each cluster node before proceeding to configure the shared storage disks.
Figure 1: Example of a two-node cluster
Configuring a Private Network Adapter
Complete these steps on the first node of your cluster.
- My network environment and select a team Properties.
- Click right click mouse on the icon.
Note: Which network adapter will serve a private network and which public one depends on physical connection network cables. IN this document we will assume that the first adapter (Connection via local network) is connected to the public network, and the second adapter (Local Area Connection 2) is connected to the cluster's private network. In your case this may not be the case.
- State. Window Status LAN Connection 2 shows the connection status and its speed. If the connection is in the disconnected state, check the cables and connections. Fix the problem before continuing. Click the button Close.
- Right click on the icon again LAN connection 2, select a command Properties and press the button Tune.
- Select a tab Additionally. The window shown in Figure 2 will appear.
- For private network network adapters, the speed must be set manually instead of the default value. Specify your network speed in the drop-down list. Don't use values "Auto Sense" or "Auto Select" to select the speed, since some network adapters may drop packets while determining the connection speed. To set the network adapter speed, specify the actual value for the parameter Connection type or Speed.
Figure 2: Additional network adapter settings
All cluster network adapters connected to the same network must be configured identically and use the same parameter values Duplex mode, Flow control, Connection type, etc. Even if different network equipment is used on different nodes, the values of these parameters must be the same.
- Select Internet Protocol (TCP/IP) in the list of components used by the connection.
- Click the button Properties.
- Set the switch to position Use the following IP address and enter the address 10.1.1.1 . (For the second node, use the address 10.1.1.2 ).
- Set the subnet mask: 255.0.0.0 .
- Click the button Additionally and select a tab WINS. Set the switch value to position Disable NetBIOS over TCP/IP. Click OK to return to the previous menu. Perform this step only for the private network adapter.
Your dialog box should look like Figure 3.
Figure 3: Private Network Connection IP Address
Configuring a public network adapter
Note: If a DHCP server is running on a public network, the IP address for the network adapter on the public network can be assigned automatically. However, this method is not recommended for cluster node adapters. We strongly recommend that you prescribe permanent IP addresses for all public and private host network adapters. Otherwise, if the DHCP server fails, access to the cluster nodes may be impossible. If you are forced to use DHCP for network adapters on a public network, use long address lease periods - this will ensure that the dynamically assigned address will remain valid even if the DHCP server is temporarily unavailable. Always assign permanent IP addresses to private network adapters. Remember that the Cluster service can only recognize one network interface per subnet. If you need help with appointments network addresses in Windows 2000, refer to the built-in help operating system.
Renaming network connections
For clarity, we recommend changing the names of your network connections. For example, you can change the name of the connection LAN connection 2 on . This method will help you more easily identify networks and correctly assign their roles.
- Right click on the icon 2.
- IN context menu select team Rename.
- Enter Connect to the cluster's private network in the text field and press the key ENTER.
- Repeat steps 1-3 and change the connection name LAN connection on Connect to a public network.
Figure 4: Renamed network connections
- The renamed network connections should look like Figure 4. Close the window Network and remote network access. New network connection names are automatically replicated to other nodes in the cluster when they are turned on.
Checking network connections and name resolutions
To check the operation of the configured network equipment, follow these steps for all network adapters on each node. To do this, you must know the IP addresses of all network adapters in the cluster. You can get this information by running the command ipconfig on each node:
- Click the button Start, select team Execute and type the command cmd in the text window. Click OK.
- Type the command ipconfig /all and press the key ENTER. You will see information about the IP protocol configuration for each network adapter on the local machine.
- In case your window is not yet open command line, follow step 1.
- Type the command ping ipaddress Where ipaddress is the IP address of the corresponding network adapter on another node. For example, assume that the network adapters have the following IP addresses:
| Node number | Name network connection | Network adapter IP address |
| 1 | Connecting to a public network | 172.16.12.12 |
| 1 | Connect to the cluster's private network | 10.1.1.1 |
| 2 | Connecting to a public network | 172.16.12.14 |
| 2 | Connect to the cluster's private network | 10.1.1.2 |
In this example you need to run the commands ping 172.16.12.14 And ping 10.1.1.2 from node 1, and execute the commands ping 172.16.12.12 And ping 10.1.1.1 from node 2.
To check name resolution, run the command ping, using the computer's name as an argument instead of its IP address. For example, to check name resolution for the first cluster node named hq-res-dc01, run the command ping hq-res-dc01 from any client computer.
Checking domain membership
All cluster nodes must be members of the same domain and have networking capabilities with a domain controller and DNS server. Nodes can be configured as member domain servers or as controllers of the same domain. If you decide to make one of the nodes a domain controller, then all other nodes in the cluster must also be configured as domain controllers of the same domain. This guide assumes that all hosts are domain controllers.
Note: For links to additional documentation on configuring domains, DNS, and DHCP services in Windows 2000, see Related Resources at the end of this document.
- Right click My computer and select a team Properties.
- Select a tab Network identification. In the dialog box Properties of the system You will see the full computer and domain name. In our example, the domain is called reskit.com.
- If you have configured the node as a member server, then at this stage you can join it to the domain. Click the button Properties and follow the instructions to join the computer to the domain.
- Close the windows Properties of the system And My computer.
Creation account cluster services
For the Cluster service, you must create a separate domain account under which it will be launched. The installer will require you to enter credentials for the Cluster service, so an account must be created before installing the service. The account must not be owned by any domain user, and must be used solely for running the Cluster service.
- Click the button Start, select a command Programs / Administration, run the snap-in.
- Expand category reskit.com, if it has not yet been deployed
- Select from the list Users.
- Right click on Users, select from the context menu Create, select User.
- Enter a name for the cluster service account as shown in Figure 5 and click Further.
Figure 5: Adding a Cluster User
- Check the boxes Prevent user from changing password And The password has no expiration date. Click the button Further and a button Ready to create a user.
Note: If your administrative security policy does not allow passwords that never expire, you will need to update the password and configure the Cluster service on each node before it expires.
- Click right key mouse on user Cluster in the right toolbar Active Directory– users and computers.
- In the context menu, select the command Add members to a group.
- Select group Administrators and press OK. The new account now has administrator privileges on local computer.
- Close the snap Active Directory - Users and Computers.
Configuring shared storage drives
Warning: Ensure that at least one of the cluster nodes is running the Windows 2000 Advanced Server or Windows 2000 Datacenter Server operating system and that the Cluster service is configured and running. Only after this can the Windows 2000 operating system be loaded on the remaining nodes. If these conditions are not met, the cluster disks may become damaged.
To begin setting up shared storage drives, turn off all nodes. After that, turn on the shared storage device, then turn on node 1.
Quorum disk
The quorum disk is used to store checkpoints and recovery log files of the cluster database, providing cluster management. We make the following recommendations for creating a quorum disk:
- Create a small partition (at least 50 MB in size) to use as a quorum disk. We generally recommend creating a quorum disk of 500 MB in size.
- Dedicate a separate disk for the quorum resource. Because if a quorum disk fails, the entire cluster will fail, we strongly recommend using a hardware disk RAID array.
During the Cluster service installation process, you will be required to assign a letter to the quorum drive. In our example we will use the letter Q.
Configuring shared storage drives
- Right click My computer, select a command Control. In the window that opens, expand the category Storage devices.
- Select a team Disk management.
- Make sure that all shared storage drives are formatted as NTFS and have the status Basic. If you connect new disk, will start automatically Disk Signing and Update Wizard. When the wizard starts, click the button Update, to continue its operation, after this the disk will be identified as Dynamic. To convert the disk to basic, right-click on Disk #(Where # – number of the disk you are working with) and select the command Revert to basic disk.
Right click area Not distributed next to the corresponding disk.
- Select a team Create a section
- Will start Partition Creation Wizard. Press the button twice Further.
- Enter the desired partition size in megabytes and click the button Further.
- Click the button Further, accepting the default drive letter suggested
- Click the button Further to format and create a partition.
Assigning drive letters
After the data bus, disks, and shared storage partitions are configured, you must assign drive letters to all partitions on all disks in the cluster.
Note: Connection points are functionality file system that allows you to install file system using existing directories, without assigning a drive letter. Mount points are not supported by clusters. Any external drive used as a cluster resource must be partitioned into NTFS partitions, and these partitions must be assigned drive letters.
- Right-click the desired partition and select Changing the drive letter and drive path.
- Select a new drive letter.
- Repeat steps 1 and 2 for all shared storage drives.
Figure 6: Disk partitions with assigned letters
- At the end of the procedure, the snap window Computer management should look like Figure 6. Close the snap-in Computer management.
- Click the button Start, select Programs / Standard, and run the program " Notebook".
- Type a few words and save the file under the name test.txt by selecting the command Save as from the menu File. Close Notebook.
- Double-click on the icon My Documents.
- Right click on the file test.txt and in the context menu select the command Copy.
- Close the window.
- Open My computer.
- Double-click the shared storage drive partition.
- Right click and select command Insert.
- A copy of the file should appear on the shared storage drive test.txt.
- Double click on the file test.txt to open it from a shared storage drive. Close the file.
- Select the file and press the key Del to delete a file from the cluster disk.
Repeat the procedure for all disks in the cluster to ensure that they are accessible from the first node.
Now turn off the first node, turn on the second node and repeat the steps in section Checking operation and sharing of disks. Follow these same steps on all additional nodes. Once you are sure that all nodes can read and write information to the shared storage drives, turn off all nodes except the first one and continue to the next section.
Press center
Creating a cluster based on Windows 2000/2003. Step by step
A cluster is a group of two or more servers that work together to provide trouble-free operation of a set of applications or services and are perceived by the client as a single unit. Cluster nodes are connected to each other using network hardware, shared resources, and server software.
Microsoft Windows 2000/2003 supports two clustering technologies: Network Load Balancing clusters and server clusters.
In the first case (load-balanced clusters), the Network Load Balancing service makes services and applications highly reliable and scalable by combining up to 32 servers into a single cluster. In this case, requests from clients are distributed among the cluster nodes in a transparent manner. If a node fails, the cluster automatically changes its configuration and switches the client to any of the available nodes. This cluster configuration mode is also called active-active mode, where one application runs on multiple nodes.
A server cluster distributes its load among the servers in the cluster, with each server carrying its own load. If a node in the cluster fails, applications and services configured to run in the cluster are transparently restarted on any of the free nodes. Server clusters use shared disks to share data within the cluster and to provide transparent access to cluster applications and services. They require special equipment, but this technology provides a very high level of reliability, since the cluster itself does not have any single point of failure. This cluster configuration mode is also called active-passive mode. An application in a cluster runs on a single node with shared data located on external storage.
The cluster approach to organizing an internal network provides the following advantages:
High Availability That is, if a service or application fails on a cluster node that is configured to work together in a cluster, the cluster software allows the application to restart on another node. Users will experience a short delay when performing some operation or will not notice a server failure at all. Scalability For applications running in a cluster, adding servers to a cluster means increasing capabilities: fault tolerance, load distribution, etc. Manageability Administrators, using a single interface, can manage applications and services, set a reaction to failure in a cluster node, distribute the load among nodes cluster and remove the load from the nodes to carry out maintenance work.
In this article I will try to collect my experience in creating cluster systems based on Windows and give a little step by step guide to create a two-node server cluster with shared data storage.
Software requirements
- Microsoft Windows 2000 Advanced (Datacenter) Server or Microsoft Windows 2003 Server Enterprise Edition installed on all servers in the cluster.
- Installed DNS service. Let me explain a little. If you are building a cluster based on two domain controllers, then it is much more convenient to use the DNS service, which you install when creating Active Directory anyway. If you are creating a cluster based on two servers that are members of a Windows NT domain, then you will have to use either the WINS service or enter the matching names and addresses of the machines in the hosts file.
- Terminal Services for remote control servers. Not necessary, but if you have Terminal Services, it’s convenient to manage servers from your workplace.
Hardware Requirements
- It is better to select hardware for a cluster node based on the Cluster Service Hardware Compatible List (HCL). Microsoft recommends that hardware be tested for compatibility with Cluster Services.
- Accordingly, you will need two servers with two network adapters each; A SCSI adapter that has an external interface for connecting an external data array.
- External array having two external interface. Each cluster node is connected to one of the interfaces.
Comment: To create a two-node cluster it is not at all necessary to have two absolutely identical servers. After a failure on the first server, you will have some time to analyze and restore the operation of the main node. The second node will work for the reliability of the system as a whole. However, this does not mean that the second server will be idle. Both cluster nodes can calmly go about their business and solve different problems. But we can configure a certain critical resource to work in a cluster, increasing its (this resource’s) fault tolerance.
Network settings requirements
- A unique NetBIOS name for the cluster.
- Five unique static IP addresses. Two for network adapters per cluster network, two for network adapters per shared network, and one for the cluster.
- Domain account for the Cluster service.
- All cluster nodes must be either member servers in the domain or domain controllers.
- Each server must have two network adapters. One for connecting to a common network (Public Network), the second for exchanging data between cluster nodes (Private Network).
Comment: According to Microsoft recommendations, your server must have two network adapters, one for the general network, the second for data exchange within the cluster. Is it possible to build a cluster on one interface - probably yes, but I haven’t tried it.
Cluster installation
When designing a cluster, you must understand that by using the same physical network for both cluster communication and the local network, you increase the failure rate of the entire system. Therefore, it is highly desirable for cluster data exchange to use one subnet, allocated as a separate physical element of the network. And for a local network you should use a different subnet. Thus, you increase the reliability of the entire system as a whole.
In case of building a two-node cluster, one switch is used common network. Two cluster servers can be connected directly to each other using a crossover cable, as shown in the figure.
Installing a two-node cluster can be divided into 5 steps
- Installing and configuring nodes in a cluster.
- Installing and configuring a shared resource.
- Checking the disk configuration.
- Configuring the first cluster node.
- Configuring the second node in the cluster.
This step-by-step guide will help you avoid mistakes during installation and save a lot of time. So, let's begin.
Installation and configuration of nodes
We'll simplify the task a little. Since all cluster nodes must be either domain members or domain controllers, we will make the first cluster node the root holder of the AD (Active Directory) directory, and the DNS service will run on it. The 2nd node of the cluster will be a full domain controller.
I am ready to skip installing the operating system, believing that you should not have any problems with this. And here is the configuration network devices I would like to explain.
Network settings
Before you begin installing the cluster and Active Directory, you must complete the network settings. I would like to divide all network settings into 4 stages. To recognize names on the network, it is advisable to have a DNS server with already existing records about cluster servers.
Each server has two network cards. One network card will serve for data exchange between cluster nodes, the second will work for clients in our network. Accordingly, we will call the first Private Cluster Connection, the second we will call Public Cluster Connection.
The network adapter settings for one and the other server are identical. Accordingly, I will show how to configure the network adapter and give a sign with network settings all 4 network adapters on both cluster nodes. To configure the network adapter, you must complete the following steps:
- My Network Places → Properties
Private Cluster Connection → Properties → Configure → Advanced
This point requires some clarification. The fact is that, according to Microsoft’s strong recommendations, all network adapters of cluster nodes must be set to the optimal speed of the adapter, as shown in the following figure.
- Internet Protocol (TCP/IP) → Properties → Use the following IP: 192.168.30.1
(For the second node, use the address 192.168.30.2). Enter the subnet mask 255.255.255.252. Use 192.168.100.1 as the DNS server address for both nodes.
- Additionally, on the Advanced → WINS tab, select Disabled NetBIOS over TCP/IP. To configure network adapters for the public network, omit this item.
- Do the same with the network card for the Public Cluster Connection local network. Use the addresses given in the table. The only difference in the configuration of the two network cards is that Public Cluster Connection does not require turning off the WINS - NetBIOS over TCP/IP mode.
To configure all network adapters on cluster nodes, use the following label:
| Knot | Network name | IP address | MASK | DNS Server |
|---|---|---|---|---|
| 1 | Public Cluster Connection | 192.168.100.1 | 255.255.255.0 | 192.168.100.1 |
| 1 | Private Cluster Connection | 192.168.30.1 | 255.255.255.252 | 192.168.100.1 |
| 2 | Public Cluster Connection | 192.168.100.2 | 255.255.255.0 | 192.168.100.1 |
| 3 | Private Cluster Connection | 192.168.30.2 | 255.255.255.252 | 192.168.100.1 |
Installing Active Directory
Since my article is not intended to talk about installing Active Directory, I will omit this point. There are all kinds of recommendations, quite a lot of books have been written about this. You are taking Domain name, like mycompany.ru, install Active Directory on the first node, add the second node to the domain as a domain controller. When you're done, check the server configurations and Active Directory.
Installing Cluster User Account
- Start → Programs → Administrative Tools → Active Directory Users and Computers
- Add a new user, for example ClusterService.
- Check the boxes for: User Cannot Change Password and Password Never Expires.
- Also add this user to the Administrators group and give him Log on as a service rights (rights are assigned in Local Security Policy and Domain Controller Security Policy).
Setting up an external data array
To configure an external data array in a cluster, you must remember that before installing the Cluster Service on the nodes, you must first configure the disks on the external array, only then install the cluster service first on the first node, and only then on the second. If the installation order is not correct, you will fail and you will not achieve your goal. Will it be possible to fix it - probably yes. When an error appears, you will have time to adjust your settings. But Microsoft is such a mysterious thing that you don’t really know what mistakes you’ll step on. It's easier to have in front of your eyes step by step instructions and don't forget to press the buttons. The steps for configuring an external array look like this:
- Both servers must be turned off, the external array must be turned on, connected to both servers.
- We turn on the first server. We gain access to the disk array.
- We check that the external disk array was created as Basic. If this is not the case, then convert the disk using the Revert to Basic Disk option.
- We create on external drive through Computer Management → Disk Management a small section. According to Microsoft recommendations, it should be at least 50 MB. I recommend creating a 500 MB partition. or a little more. This is quite enough to host clustered data. The partition must be formatted in NTFS.
- On both cluster nodes, this partition will be named with the same letter, for example, Q. Accordingly, when creating a partition on the first server, select the item Assign the following drive letter - Q.
- You can mark up the rest of the disk as you wish. Of course, it is highly advisable to use a file NTFS system. For example, when setting up DNS and WINS services, the main service databases will be moved to shared drive(not the Q system volume, but the second one you created). And for security reasons, it will be more convenient for you to use NTFS volumes.
- Close Disk Management and check access to the newly created partition. For example, you can create on it text file test.txt , write and delete. If everything went well, then we are done with the configuration of the external array on the first node.
- Now we turn off the first server. The external array must be enabled. We turn on the second server and check access to the created partition. We will also check that the letter assigned to the first partition is identical to the one we selected, that is, Q.
This completes the configuration of the external array.
Installing Cluster Service Software
Configuration of the first cluster node
Before installing Cluster Service Software, all cluster nodes must be turned off and all external arrays must be turned on. Let's move on to the configuration of the first node. The external array is turned on, the first server is turned on. The entire installation process occurs using the Cluster Service Configuration Wizard:
Configuration of the second cluster node
To install and configure the second cluster node, the first node must be turned on, all network drives were included. The procedure for setting up the second node is very similar to what I described above. However there is minor changes. To do this, use the following instructions:
- In the Create or Join a Cluster dialog box, select The second or next node in the cluster and click next.
- Enter the cluster name that we set earlier (in the example it is MyCluster) and click next.
- After connecting the second node to the cluster, Cluster Service Configuration Wizard will automatically take all settings from the main node. To start the Cluster Service, use the name we created earlier.
- Enter your account password and click next.
- In the next dialog box, click Finish to complete the installation.
- Cluster service will be launched on the second node.
- Close the Add/Remove Programs window.
To install additional cluster nodes, use the same instructions.
Postscript, thanks
To avoid getting confused with all the stages of installing a cluster, I will provide a small table that reflects all the main stages.
| Step | Node 1 | Node 2 | External array |
|---|
Loading...
