Thursday, June 28, 2018

Downgrade MongoDB - Enterprise to Community Edition


MongoDB Enterprise to Community Edition  - Implementation

Overview:

Shards 1(rs_0)

tommy1:27019 - Primary
tommy2:27019 - Secondary
tommy3:27019 - Secondary

Shards 2(rs_1)

tommy4:27019 - Primary
tommy5:27019 - Secondary
tommy6:27019 - Secondary

Shards 3(rs_3)

tommy7:27019 - Primary
tommy8:27019 - Secondary
tommy9:27019 - Secondary

ConfigDB:

tommy10:27018
tommy11:27018
tommy12:27018

Mongos:

tommy13:27017

Version : 3.0.5

Cluster size : 56.3GB

Implementation Plan:

1) Login as mongod user

Sudo -u mongod -i
Password

Note: just for reference

Existing path in Prod server bin - /var/lib/mongodb-mms-automation/mongodb-linux-x86_64-3.0.5-ent/bin/
Existing path in Prod server config - /data/automation-mongod.conf

Generic Path for bin - /usr/mongodb_3.0.5/bin/
Generic Path for config - /usr/mongodb_3.0.5/conf/

2) Create necessary directories in all the servers

# mkdir -p /usr/mongodb_3.0.5/bin/
# chmod 775 /usr/mongodb_3.0.5/bin/

# mkdir /usr/mongodb_3.0.5/conf/
# chmod 775 /usr/mongodb_3.0.5/conf/

3) Copy the New Binaries to all the servers in bin folder from - tommy13

Path - /usr/mongodb_3.0.5/bin/

Log in to tommy13 – run the below command in all the servers
# scp /tmp/mongodb-linux-x86_64-3.0.5.tgz [user account]@[hostname]:/tmp/

Then log in to each server, run the commands – ( have to run this in all servers)

# cd /tmp
# tar -zxvf mongodb-linux-x86_64-3.0.5.tgz
# cp -R /tmp/mongodb-linux-x86_64-3.0.5/* /usr/mongodb_3.0.5/

4) Copy the existing configuration files to the new path

Path - /usr/mongodb_3.0.5/conf/

For tommy13 - mongos
cp /var/lib/mongodb-mms-automation/workspace/mongos-cluster_mongos_0.conf /usr/mongodb_3.0.5/conf/

For tommy10 - config
cp /data/cluster_config_10/automation-mongod.conf /usr/mongodb_3.0.5/conf/
For tommy11 - config
cp /data/cluster_config_11/automation-mongod.conf /usr/mongodb_3.0.5/conf/

For tommy12 - config
cp /data/cluster_config_12/automation-mongod.conf /usr/mongodb_3.0.5/conf/

For tommy1 - primary
cp /data/cluster_rs_0_2/automation-mongod.conf /usr/mongodb_3.0.5/conf/

For tommy2 - secondary
cp /data/cluster_rs_0_1/automation-mongod.conf /usr/mongodb_3.0.5/conf/

For tommy3 - secondary
cp /data/cluster_rs_0_3/automation-mongod.conf /usr/mongodb_3.0.5/conf/

For tommy4 - primary
cp /data/cluster_rs_0_4/automation-mongod.conf /usr/mongodb_3.0.5/conf/

For tommy5 - secondary
cp /data/cluster_rs_1_4/automation-mongod.conf /usr/mongodb_3.0.5/conf/

For tommy6 - seconday
cp /data/cluster_rs_1_5/automation-mongod.conf /usr/mongodb_3.0.5/conf/

For tommy7 - primary
cp /data/cluster_rs_2_8/automation-mongod.conf /usr/mongodb_3.0.5/conf/

For tommy8 - secondary
cp /data/cluster_rs_2_7/automation-mongod.conf /usr/mongodb_3.0.5/conf/

For tommy9 secondary
cp /data/cluster_rs_2_9/automation-mongod.conf /usr/mongodb_3.0.5/conf/

5) Rename the config file names

tommy13 - mongos
# mv /usr/mongodb_3.0.5/conf/mongos-cluster_mongos_0.conf /usr/mongodb_3.0.5/conf/mongos.conf

tommy1, tommy2, tommy3, tommy4, 
tommy5, tommy6, tommy7, tommy8,
tommy9 - mongod
# mv /usr/mongodb_3.0.5/conf/automation-mongod.conf /usr/mongodb_3.0.5/conf/mongod.conf

tommy10, tommy11, tommy12 - config
# mv /usr/mongodb_3.0.5/conf/automation-mongod.conf /usr/mongodb_3.0.5/conf/mongod_config.conf

6) Open all config files in all servers and delete the first 2 lines which is hashed (#)

tommy13 - mongos
# vi /usr/mongodb_3.0.5/conf/mongos.conf

tommy1, tommy2, tommy3, tommy4, 
tommy5, tommy6, tommy7, tommy8,
tommy9 - mongod
# vi /usr/mongodb_3.0.5/conf/mongod.conf

tommy10, tommy11, tommy12 - config
# vi /usr/mongodb_3.0.5/conf/mongod_config.conf

6) Disable the balancer

> login to tommy13 server
> connect to mongos
> mongo admin --port 27017
# mongos> sh.getBalancerState()
true
# mongos> sh.stopBalancer()
# mongos> sh.getBalancerState()
false

7) Downgrade secondary members of the replica set in each shard 1

1st secondary

login to tommy2 server
check the process
# ps aux | grep mongo
Remove MongoDB automation agent
# sudo rpm -e mongodb-mms-automation-agent-manager
connect to mongo
# mongo admin --port 27019
# db.shutdownServer()
start the server with the new binaries and new config file
# /usr/mongodb_3.0.5/bin/mongod -f /usr/mongodb_3.0.5/conf/mongod.conf
connect to mongo and wait for the member to recover to SECONDARY state before downgrading the next secondary - to check
# rs.status()

2nd secondary

login to tommy3 server
check the process
# ps aux | grep mongo
Remove MongoDB automation agent
# sudo rpm -e mongodb-mms-automation-agent-manager
connect to mongo
# mongo admin --port 27019
# db.shutdownServer()
start the server with the new binaries and new config file
# /usr/mongodb_3.0.5/bin/mongod -f /usr/mongodb_3.0.5/conf/mongod.conf
connect to mongo and wait for the member to recover to SECONDARY state before downgrading the next secondary - to check
# rs.status()

8) Step down the primary in shard 1.

login to tommy1 server
check the process
# ps aux | grep mongo
Remove MongoDB automation agent
# sudo rpm -e mongodb-mms-automation-agent-manager
connect to mongo
# mongo admin --port 27019
Step down the primary
# rs.stepDown()
when the server become secondary
# use admin
# db.shutdownServer()
start the server with the new binaries and new config file
# /var/lib/mongodb/mongodb_3.0.5/bin/mongod -f /var/lib/mongodb/conf/mongod.conf
connect to mongo and wait for the member to recover to SECONDARY state before downgrading the next secondary - to check
# rs.status()

9) Downgrade secondary members of the replica set in each shard 2

1st secondary

login to tommy5 server
check the process
# ps aux | grep mongo
Remove MongoDB automation agent
# sudo rpm -e mongodb-mms-automation-agent-manager
connect to mongo
# mongo admin --port 27019
# db.shutdownServer()
start the server with the new binaries and new config file
# /usr/mongodb_3.0.5/bin/mongod -f /usr/mongodb_3.0.5/conf/mongod.conf
connect to mongo and wait for the member to recover to SECONDARY state before downgrading the next secondary - to check
# rs.status()

2nd secondary

login to tommy6 server
check the process
# ps aux | grep mongo
Remove MongoDB automation agent
# sudo rpm -e mongodb-mms-automation-agent-manager
connect to mongo
# mongo admin --port 27019
# db.shutdownServer()
start the server with the new binaries and new config file
# /usr/mongodb_3.0.5/bin/mongod -f /usr/mongodb_3.0.5/conf/mongod.conf
connect to mongo and wait for the member to recover to SECONDARY state before downgrading the next secondary - to check
# rs.status()

10) Step down the primary in shard 2.

login to tommy4 server
check the process
# ps aux | grep mongo
Remove MongoDB automation agent
# sudo rpm -e mongodb-mms-automation-agent-manager
connect to mongo
# mongo admin --port 27019
Step down the primary
# rs.stepDown()
when the server become secondary
# use admin
# db.shutdownServer()
start the server with the new binaries and new config file
# /var/lib/mongodb/mongodb_3.0.5/bin/mongod -f /var/lib/mongodb/conf/mongod.conf
connect to mongo and wait for the member to recover to SECONDARY state before downgrading the next secondary - to check
# rs.status()

11) Downgrade secondary members of the replica set in each shard 3

1st secondary

login to tommy8 server
check the process
# ps aux | grep mongo
Remove MongoDB automation agent
# sudo rpm -e mongodb-mms-automation-agent-manager
connect to mongo
# mongo admin --port 27019
# db.shutdownServer()
start the server with the new binaries and new config file
# /usr/mongodb_3.0.5/bin/mongod -f /usr/mongodb_3.0.5/conf/mongod.conf
connect to mongo and wait for the member to recover to SECONDARY state before downgrading the next secondary - to check
# rs.status()

2nd secondary

login to tommy9 server
check the process
# ps aux | grep mongo
Remove MongoDB automation agent
# sudo rpm -e mongodb-mms-automation-agent-manager
connect to mongo
# mongo admin --port 27019
# db.shutdownServer()
start the server with the new binaries and new config file
# /usr/mongodb_3.0.5/bin/mongod -f /usr/mongodb_3.0.5/conf/mongod.conf
connect to mongo and wait for the member to recover to SECONDARY state before downgrading the next secondary - to check
# rs.status()

12) Step down the primary in shard 3.

login to tommy7 server
check the process
# ps aux | grep mongo
Remove MongoDB automation agent
# sudo rpm -e mongodb-mms-automation-agent-manager
connect to mongo
# mongo admin --port 27019
Step down the primary
# rs.stepDown()
when the server become secondary
# use admin
# db.shutdownServer()
start the server with the new binaries and new config file
# /var/lib/mongodb/mongodb_3.0.5/bin/mongod -f /var/lib/mongodb/conf/mongod.conf
connect to mongo and wait for the member to recover to SECONDARY state before downgrading the next secondary - to check
# rs.status()

13) Downgrade the SCCC config servers in reverse order

> Reverse order based on the configDB option in mongos config file
configDB: tommy10:27018, tommy11:27018, tommy12:27018
> Downgrade Config3, followed by config2 and then config1 so
> Downgrade tommy12, followed by tommy11 and then tommy10

tommy12

login to tommy12 server
check the process
# ps aux | grep mongo
Remove MongoDB automation agent
# sudo rpm -e mongodb-mms-automation-agent-manager
connect to mongo
# mongo admin --port 27018
# db.shutdownServer()
start the server with the new binaries and new config file
# /usr/mongodb_3.0.5/bin/mongod -f /usr/mongodb_3.0.5/conf/mongod_config.conf
connect to mongo and wait for the member to recover to SECONDARY state before downgrading the next secondary - to check
# rs.status()

tommy11

login to tommy11 server
check the process
# ps aux | grep mongo
Remove MongoDB automation agent
# sudo rpm -e mongodb-mms-automation-agent-manager
connect to mongo
# mongo admin --port 27018
# db.shutdownServer()
start the server with the new binaries and new config file
# /usr/mongodb_3.0.5/bin/mongod -f /usr/mongodb_3.0.5/conf/mongod.conf
connect to mongo and wait for the member to recover to SECONDARY state before downgrading the next secondary - to check
# rs.status()

tommy10

login to tommy10 server
check the process
# ps aux | grep mongo
Remove MongoDB automation agent
# sudo rpm -e mongodb-mms-automation-agent-manager
connect to mongo
# mongo admin --port 27018
# db.shutdownServer()
start the server with the new binaries and new config file
# /usr/mongodb_3.0.5/bin/mongod -f /usr/mongodb_3.0.5/conf/mongod.conf
connect to mongo and wait for the member to recover to SECONDARY state before downgrading the next secondary - to check
# rs.status()

14) Downgrade the mongos instances.

login to tommy13 server
check the process
# ps aux | grep mongo
Remove MongoDB automation agent
# sudo rpm -e mongodb-mms-automation-agent-manager
connect to mongo
# mongo admin --port 27017
# db.shutdownServer()
start the server with the new binaries and new config file
# /usr/mongodb_3.0.5/bin/mongos -f /usr/mongodb_3.0.5/conf/mongos.conf
connect to mongo and wait for the member to recover to SECONDARY state before downgrading the next secondary - to check
# rs.status()


15) Re-enable the balancer.

- Once the downgrade of sharded cluster components is complete, re-enable the balancer.
- login to tommy13 server
- connect to mongo
# mongo --port 27017
# mongos> sh.getBalancerState()
false
# mongos> sh.startBalancer()
# mongos> sh.getBalancerState()
true

Note: This implementation is done before the migration of Config Server from SCCC to ReplicaSet


2 comments:

  1. what are the risk of problem that could be with running application which was earlier using enterprise edition and now downgraded to community? I mean any drawbacks?

    ReplyDelete