概念:
MongoDB 分片是使用多个服务器存储数据的方法, 以支持巨大的数据存储和对数据进行存储
优势:
1, 减少了每个分片需啊哟处理的请求数, 群集可以提高自己的存储容量和吞吐量
2, 减少了每个分片存储的数据
三个主要组件:
shard: 分片服务器, 用于存储实际的数据块, 由多台服务器组成一个复制集承担, 防止主机单点故障
config server: 配置服务器, 存储整个分片群集的配置信息, 包括块信息
routers: 前端路由, 客户端由此进入, 让整个群集看上去像单一数据库
如何部署 MongoDB 分片群集!!!
1, 安装 MongoDB3.2 版本包和 openssl-devel 包
- [root@localhost ~]# yum install openssl-devel -y
- [root@localhost ~]# mkdir /abc
- [root@localhost ~]# mount.cifs //192.168.200.1/orc /abc
- Password for root@//192.168.200.1/orc:
- [root@localhost ~]# cd /abc
- [root@localhost abc]# ls
- mongodb-linux-x86_64-3.2.1.tgz //MongoDB 3.2 版本
- [root@localhost abc]# tar zxvf mongodb-linux-x86_64-3.2.1.tgz -C /opt
- [root@localhost abc]# cd /opt
- [root@localhost opt]# ls
- mongodb-linux-x86_64-3.2.1 rh
- [root@localhost opt]# mv mongodb-linux-x86_64-3.2.1//usr/local/mongodb
- [root@localhost opt]# cd /usr/local/bin/
- [root@localhost bin]# ln -s /usr/local/mongodb/bin/mongo /usr/bin/mongo // 建立软连接
- [root@localhost bin]# ln -s /usr/local/mongodb/bin/mongod /usr/bin/mongod
- [root@localhost bin]# mkdir -p /data/mongodb/mongodb{1,2,3,4}
- [root@localhost bin]# mkdir /data/mongodb/logs
- [root@localhost bin]# touch /data/mongodb/logs/mongodb{1,2,3,4}.log
- [root@localhost bin]# chmod -R 777 /data/mongodb/logs/*.log
- [root@localhost bin]# ulimit -n 25000 // 打开文件数量
- [root@localhost bin]# ulimit -u 25000 // 进程并发数
2, 部署配置服务器
编辑 mongodb1.conf 配置文件, 端口为 37017, 设置 configsvr=true, 启动配置服务器
- [root@localhost bin]# vim mongodb1.con
- port=37017
- dbpath=/data/mongodb/mongodb1
- logpath=/data/mongodb/logs/mongodb1.log
- logappend=true
- fork=true
- maxConns=5000
- storageEngine=mmapv1
- configsvr=true // 开启配置服务器
3, 某节点内存不足, 从其它节点分配内存
- [root@localhost bin]# sysctl -w vm.zone_reclaim_mode=0
- vm.zone_reclaim_mode = 0
- [root@localhost bin]# echo never> /sys/kernel/mm/transparent_hugepage/enabled
- [root@localhost bin]# echo never> /sys/kernel/mm/transparent_hugepage/defrag
- [root@localhost bin]# mongod -f mongodb1.conf
- about to fork child process, waiting until server is ready for connections.
- forked process: 6895
- child process started successfully, parent exiting
4, 部署分片服务器
编辑 mongodb2.conf 配置文件, 端口为 47017, 设置 shardsvr=true; 编辑 mongodb3.conf 配置文件, 端口为 47018, 启动两个分片服务器
- [root@localhost bin]# cp -p mongodb1.conf mongodb2.conf
- [root@localhost bin]# vim mongodb2.conf
- port=47017 // 修改端口
- dbpath=/data/mongodb/mongodb2
- logpath=/data/mongodb/logs/mongodb2.log
- logappend=true
- fork=true
- maxConns=5000
- storageEngine=mmapv1
- shardsvr=true // 开启分片服务器
- [root@localhost bin]# cp -p mongodb2.conf mongodb3.conf
- [root@localhost bin]# vim mongodb3.conf
- port=47018
- dbpath=/data/mongodb/mongodb3
- logpath=/data/mongodb/logs/mongodb3.log
- logappend=true
- fork=true
- maxConns=5000
- storageEngine=mmapv1
- shardsvr=true
- [root@localhost bin]# mongod -f mongodb2.conf
- about to fork child process, waiting until server is ready for connections.
- forked process: 7080
- child process started successfully, parent exiting
- [root@localhost bin]# mongod -f mongodb3.conf
- about to fork child process, waiting until server is ready for connections.
- forked process: 7107
- child process started successfully, parent exiting
- [root@localhost bin]# netstat -antp | grep mongod
- tcp 0 0 0.0.0.0:37017 0.0.0.0:* LISTEN god
- tcp 0 0 0.0.0.0:47017 0.0.0.0:* LISTEN god
- tcp 0 0 0.0.0.0:47018 0.0.0.0:* LISTEN god
5, 启动路由服务器
./mongos --help 命令可以查看路由相关参数信息. chunkSize 为数据块大小, 默认为 200M, 这里将值设为 1
[root@localhost bin]# ./mongos --port 27017 --fork --logpath=/usr/local/mongodb/bin/route.log --configdb 192.168.200.142:37017 --chunkSize 1
6, 启用分片服务器
- mongos> sh.addShard("192.168.200.142:47017")
- { "shardAdded" : "shard0000", "ok" : 1 }
- mongos> sh.addShard("192.168.200.142:47018")
- { "shardAdded" : "shard0001", "ok" : 1 }
- mongos> sh.status()
- --- Sharding Status ---
- sharding version: {
- "_id" : 1,
- "minCompatibleVersion" : 5,
- "currentVersion" : 6,
- "clusterId" : ObjectId("5b4f68599dcf397cc4e7c598")
- }
- shards: // 分片服务器开启
- { "_id" : "shard0000", "host" : "192.168.200.142:47017" }
- { "_id" : "shard0001", "host" : "192.168.200.142:47018" }
- active mongoses:
- "3.2.1" : 1
- balancer:
- Currently enabled: yes
- Currently running: no
- Failed balancer rounds in last 5 attempts: 0
- Migration Results for the last 24 hours:
- No recent migratio
7, 分片功能
- [root@localhost bin]# mongo
- mongos> use school
- switched to db school
- mongos> for(var i=1;i<=10000;i++)db.users.insert({"id":i,"name":"jack"+i})
- WriteResult({ "nInserted" : 1 })
- mongos> show dbs
- config 0.031GB
- school 0.078GB
- mongos> use school
- switched to db school
- mongos> show collections
- system.indexes
- users
- mongos> db.users.find().limit(10) // 查看头十行内容
- { "_id" : ObjectId("5b4f6a24ef33d588c0dbc4ae"), "id" : 1, "name" : "jack1" }
- { "_id" : ObjectId("5b4f6a24ef33d588c0dbc4af"), "id" : 2, "name" : "jack2" }
- { "_id" : ObjectId("5b4f6a24ef33d588c0dbc4b0"), "id" : 3, "name" : "jack3" }
- { "_id" : ObjectId("5b4f6a24ef33d588c0dbc4b1"), "id" : 4, "name" : "jack4" }
- { "_id" : ObjectId("5b4f6a24ef33d588c0dbc4b2"), "id" : 5, "name" : "jack5" }
- { "_id" : ObjectId("5b4f6a24ef33d588c0dbc4b3"), "id" : 6, "name" : "jack6" }
- { "_id" : ObjectId("5b4f6a24ef33d588c0dbc4b4"), "id" : 7, "name" : "jack7" }
- { "_id" : ObjectId("5b4f6a24ef33d588c0dbc4b5"), "id" : 8, "name" : "jack8" }
- { "_id" : ObjectId("5b4f6a24ef33d588c0dbc4b6"), "id" : 9, "name" : "jack9" }
- { "_id" : ObjectId("5b4f6a24ef33d588c0dbc4b7"), "id" : 10, "name" : "jack10" }
- mongos> sh.status() // 查看数据库分片信息
- --- Sharding Status ---
- sharding version: {
- "_id" : 1,
- "minCompatibleVersion" : 5,
- "currentVersion" : 6,
- "clusterId" : ObjectId("5b4f68599dcf397cc4e7c598")
- }
- shards:
- { "_id" : "shard0000", "host" : "192.168.200.142:47017" }
- { "_id" : "shard0001", "host" : "192.168.200.142:47018" }
- active mongoses:
- "3.2.1" : 1
- balancer:
- Currently enabled: yes
- Currently running: no
- Failed balancer rounds in last 5 attempts: 0
- Migration Results for the last 24 hours:
- No recent migrations
- databases:
- { "_id" : "school", "primary" : "shard0000", "partitioned" : false }
- mongos> sh.enableSharding("school") // 启用数据库分片
- { "ok" : 1 }
- mongos> sh.status()
- --- Sharding Status ---
- sharding version: {
- "_id" : 1,
- "minCompatibleVersion" : 5,
- "currentVersion" : 6,
- "clusterId" : ObjectId("5b4f68599dcf397cc4e7c598")
- }
- shards:
- { "_id" : "shard0000", "host" : "192.168.200.142:47017" }
- { "_id" : "shard0001", "host" : "192.168.200.142:47018" }
- active mongoses:
- "3.2.1" : 1
- balancer:
- Currently enabled: yes
- Currently running: no
- Failed balancer rounds in last 5 attempts: 0
- Migration Results for the last 24 hours:
- No recent migrations
- databases:
- { "_id" : "school", "primary" : "shard0000", "partitioned" : true }
- mongos> db.users.createIndex({"id":1}) // 对 users 表创建索引
- {
- "raw" : {
- "192.168.200.142:47017" : {
- "createdCollectionAutomatically" : false,
- "numIndexesBefore" : 1,
- "numIndexesAfter" : 2,
- "ok" : 1
- }
- },
- "ok" : 1
- }
- mongos> sh.shardCollection("school.users",{"id":1})// 表分片
- { "collectionsharded" : "school.users", "ok" : 1 }
- mongos> sh.status()
- --- Sharding Status ---
- sharding version: {
- "_id" : 1,
- "minCompatibleVersion" : 5,
- "currentVersion" : 6,
- "clusterId" : ObjectId("5b4f68599dcf397cc4e7c598")
- }
- shards:
- { "_id" : "shard0000", "host" : "192.168.200.142:47017" }
- { "_id" : "shard0001", "host" : "192.168.200.142:47018" }
- active mongoses:
- "3.2.1" : 1
- balancer:
- Currently enabled: yes
- Currently running: no
- Failed balancer rounds in last 5 attempts: 0
- Migration Results for the last 24 hours:
- No recent migrations
- databases:
- { "_id" : "school", "primary" : "shard0000", "partitioned" : true }
- school.users
- shard key: { "id" : 1 }
- unique: false
- balancing: true
- chunks:
- shard0000 3
- { "id" : { "$minKey" : 1 } } -->> { "id" : 4682 } on : shard0000 Timestamp(1, 0)
- { "id" : 4682 } -->> { "id" : 9364 } on : shard0000 Timestamp(1, 1)
- { "id" : 9364 } -->> { "id" : { "$maxKey" : 1 } } on : shard0000 Timestamp(1, 2) // 分片成功
来源: http://www.bubuko.com/infodetail-2691067.html