單數(shù)據(jù)庫(kù)vs多數(shù)據(jù)庫(kù),單實(shí)例vs多實(shí)例 效率測(cè)試
最近公司的項(xiàng)目準(zhǔn)備優(yōu)化一下系統(tǒng)的性能,希望在數(shù)據(jù)庫(kù)方面看有沒(méi)有提升的空間,目前壓力測(cè)試發(fā)現(xiàn)數(shù)據(jù)庫(kù)服務(wù)器壓力還不夠大,Web服務(wù)器壓力也不是很大的情況下,前臺(tái)頁(yè)面訪問(wèn)卻很慢,看有沒(méi)有辦法充分利用數(shù)據(jù)庫(kù)服務(wù)器的性能,于是做了一個(gè)單數(shù)據(jù)庫(kù),多數(shù)據(jù)庫(kù),單實(shí)例,多實(shí)例不同情況下的數(shù)據(jù)訪問(wèn)效率測(cè)試。
測(cè)試環(huán)境:
- CPU:Inter Core2 Quad,Q8300,2.50GHz;
- 內(nèi)存:4.00GB
- 系統(tǒng):Windows 7 32位系統(tǒng)
- 數(shù)據(jù)庫(kù)系統(tǒng):SqlServer 2008,有兩個(gè)實(shí)例,一個(gè)是默認(rèn)實(shí)例,一個(gè)是命名實(shí)例QE2
測(cè)試數(shù)據(jù):
67萬(wàn)真實(shí)的基金收益數(shù)據(jù),將這個(gè)表的數(shù)據(jù)放到了3個(gè)數(shù)據(jù)庫(kù)中,詳細(xì)內(nèi)容見(jiàn)下面的連接字符串配置:
- <add name ="Ins1_DB1" connectionString ="Data Source=.;Initial Catalog=TestDB;Integrated Security=True"/>
- <add name ="Ins1_DB2" connectionString ="Data Source=.;Initial Catalog=LocalDB;Integrated Security=True"/>
- <add name ="Ins2_DB" connectionString ="Data Source=.\QE2;Initial Catalog=TestDB;Integrated Security=True"/>
測(cè)試內(nèi)容:
首先篩選出表中所有的基金代碼,然后統(tǒng)計(jì)每只基金的最新收益率日期,對(duì)應(yīng)的T-SQL代碼如下:
- declare @max_fsrq datetime
- declare @currJJDM varchar(10)
- declare @temp table (jjdm2 varchar(10))
- declare @useTime datetime
- set @useTime =GETDATE ();
- insert into @temp(jjdm2)
- select jjdm from [FundYield] group by jjdm order by jjdm asc
- while EXISTS (select jjdm2 from @temp)
- begin
- set @currJJDM=(select top 1 jjdm2 from @temp)
- select @max_fsrq = MAX(fsrq) from [FundYield] where jjdm=@currJJDM
- delete from @temp where jjdm2 =@currJJDM
- print @max_fsrq
- end
- print 'T-SQL Execute Times(ms):'
- print datediff(ms,@useTime,getdate())
直接執(zhí)行這個(gè)T-SQL腳本,在數(shù)據(jù)庫(kù)表沒(méi)有索引的情況下,耗費(fèi)的時(shí)間是:
- T-SQL Execute Times(ms):
- 58796
根據(jù)這個(gè)功能,寫(xiě)了一個(gè).net控制臺(tái)程序來(lái)測(cè)試,測(cè)試程序沒(méi)有使用任何數(shù)據(jù)訪問(wèn)框架,直接使用ADO.NET,下面是多線程測(cè)試的代碼,其它代碼略:
- public static void Test2(string connName1,string connName2)
- {
- System.Diagnostics.Stopwatch watch = new System.Diagnostics.Stopwatch();
- watch.Start();
- string allJjdmList = "";
- string connString = getConnectionString();
- //SqlConnection conn = new SqlConnection(connString);
- //conn.Open();
- string sql = "select jjdm from [FundYield] group by jjdm order by jjdm asc";
- DataSet ds = getData(connString, sql);
- int allCount = ds.Tables[0].Rows.Count;
- int p = (int)(allCount * 0.5);
- System.Threading.Thread t1=new System.Threading.Thread (new System.Threading.ParameterizedThreadStart (tp1=>
- {
- for (int i = 0; i < p; i++)
- {
- string jjdm = ds.Tables[0].Rows[i][0].ToString();
- object result = getSclar(ConfigurationManager.ConnectionStrings[connName1].ConnectionString,
- string.Format("select MAX(fsrq) from [FundYield] where jjdm='{0}'", jjdm));
- if (result != DBNull.Value)
- {
- DateTime dt = Convert.ToDateTime(result);
- //Console.WriteLine("Thread 2 No {0} ,jjdm[{1}] last FSRQ is:{2}", i, jjdm, dt);
- }
- allJjdmList = allJjdmList + "," + jjdm;
- }
- Console.WriteLine("Tread 1 used all time is(ms):{0}", watch.ElapsedMilliseconds);
- }
- ));
- System.Threading.Thread t2 = new System.Threading.Thread(new System.Threading.ParameterizedThreadStart(tp2 =>
- {
- for (int i = p; i < allCount; i++)
- {
- string jjdm = ds.Tables[0].Rows[i][0].ToString();
- //這里不論使用default還是express,區(qū)別不大
- object result = getSclar(ConfigurationManager.ConnectionStrings[connName2].ConnectionString,
- string.Format("select MAX(fsrq) from [FundYield] where jjdm='{0}'", jjdm));
- if (result != DBNull.Value)
- {
- DateTime dt = Convert.ToDateTime(result);
- //Console.WriteLine("Thread 2 No {0} ,jjdm[{1}] last FSRQ is:{2}", i, jjdm, dt);
- }
- allJjdmList = allJjdmList + "," + jjdm;
- }
- Console.WriteLine("Tread 2 used all time is(ms):{0}", watch.ElapsedMilliseconds);
- }
- ));
- t1.Start();
- t2.Start();
- t1.Join();
- t2.Join();
- Console.WriteLine("====All thread completed!========");
- }
#p#
下面是測(cè)試結(jié)果:
第一次,數(shù)據(jù)庫(kù)沒(méi)有創(chuàng)建索引,進(jìn)行全表掃描:
- ------單數(shù)據(jù)庫(kù),單線程測(cè)試---------
- used all time is(ms):59916
- ------同一實(shí)例,雙數(shù)據(jù)庫(kù),單線程測(cè)試---------
- used all time is(ms):59150
- ------同一實(shí)例,雙數(shù)據(jù)庫(kù),多線程測(cè)試---------
- Tread 2 used all time is(ms):51223
- Tread 1 used all time is(ms):58175
- ====All thread completed!========
- ------雙實(shí)例,雙數(shù)據(jù)庫(kù),單線程測(cè)試---------
- used all time is(ms):58230
- ------雙實(shí)例,雙數(shù)據(jù)庫(kù),多線程測(cè)試---------
- Tread 2 used all time is(ms):52705
- Tread 1 used all time is(ms):58293
- ====All thread completed!========
第二次,數(shù)據(jù)庫(kù)響應(yīng)的字段創(chuàng)建索引,下面是測(cè)試結(jié)果:
- ------單數(shù)據(jù)庫(kù),單線程測(cè)試---------
- used all time is(ms):1721
- ------同一實(shí)例,雙數(shù)據(jù)庫(kù),單線程測(cè)試---------
- used all time is(ms):1737
- ------同一實(shí)例,雙數(shù)據(jù)庫(kù),多線程測(cè)試---------
- Tread 2 used all time is(ms):1684
- Tread 1 used all time is(ms):1714
- ====All thread completed!========
- ------雙實(shí)例,雙數(shù)據(jù)庫(kù),單線程測(cè)試---------
- used all time is(ms):1874
- ------單數(shù)據(jù)庫(kù),單線程測(cè)試---------
- used all time is(ms):1699
- ------同一實(shí)例,雙數(shù)據(jù)庫(kù),單線程測(cè)試---------
- used all time is(ms):1754
- ------同一實(shí)例,雙數(shù)據(jù)庫(kù),多線程測(cè)試---------
- Tread 1 used all time is(ms):1043
- Tread 2 used all time is(ms):1103
- ====All thread completed!========
- ------雙實(shí)例,雙數(shù)據(jù)庫(kù),單線程測(cè)試---------
- used all time is(ms):1838
- ------雙實(shí)例,雙數(shù)據(jù)庫(kù),多線程測(cè)試---------
- Tread 1 used all time is(ms):1072
- Tread 2 used all time is(ms):1139
- ====All thread completed!========
測(cè)試結(jié)論:
綜合全表掃描訪問(wèn)和有索引方式的訪問(wèn),
單線程訪問(wèn):
- 在同一個(gè)數(shù)據(jù)庫(kù)實(shí)例上,雙數(shù)據(jù)庫(kù)沒(méi)有體現(xiàn)出優(yōu)勢(shì),甚至單數(shù)據(jù)庫(kù)稍微優(yōu)勝于多數(shù)據(jù)庫(kù);
- 在兩個(gè)數(shù)據(jù)庫(kù)實(shí)例上,雙實(shí)例雙實(shí)例要落后于單實(shí)例單數(shù)據(jù)庫(kù);
多線程訪問(wèn):
- 雙數(shù)據(jù)庫(kù)實(shí)例稍微落后于單數(shù)據(jù)庫(kù)實(shí)例;
綜合結(jié)論,看來(lái)不論是雙數(shù)據(jù)庫(kù)還是雙實(shí)例,對(duì)比與單實(shí)例或者單數(shù)據(jù)庫(kù),都沒(méi)有體現(xiàn)出優(yōu)勢(shì),看來(lái)前者的優(yōu)勢(shì)不在于訪問(wèn)效率,一位朋友說(shuō),數(shù)據(jù)庫(kù)實(shí)例是不同的服務(wù),控制粒度更小,維護(hù)影響比較低。但我想,雙數(shù)據(jù)庫(kù)實(shí)例,雙數(shù)據(jù)庫(kù),多核CPU,應(yīng)該跟兩臺(tái)數(shù)據(jù)庫(kù)服務(wù)器差不多的性能吧,怎么沒(méi)有體現(xiàn)優(yōu)勢(shì)呢?也許是我的測(cè)試機(jī)器僅僅有一個(gè)磁盤(pán),這里磁盤(pán)IO成了瓶頸。
這個(gè)測(cè)試有沒(méi)有意義,或者這個(gè)結(jié)果的原因,還請(qǐng)大牛們多多指教!
意外發(fā)現(xiàn):
1,有人說(shuō)頻繁的查詢?cè)谕耆珨?shù)據(jù)庫(kù)中進(jìn)行效率最高,測(cè)試發(fā)現(xiàn),在查詢分析器上直接運(yùn)行上面的那個(gè)T-SQL腳本,跟程序從數(shù)據(jù)庫(kù)取出數(shù)據(jù),再加工計(jì)算查詢,效率上沒(méi)有明顯的區(qū)別,所以哪些支持“將復(fù)雜的業(yè)務(wù)邏輯寫(xiě)在存儲(chǔ)過(guò)程中效率最高的觀點(diǎn)是站不住腳的!” ,ADO.NET從數(shù)據(jù)庫(kù)來(lái)回操作數(shù)據(jù)一樣有效率,如果加上復(fù)雜的字符函數(shù)計(jì)算和大批量的循環(huán)操作,存儲(chǔ)過(guò)程的效率不一定高。
2,在使用程序進(jìn)行頻繁的數(shù)據(jù)庫(kù)操作的時(shí)候,使用一個(gè)連接對(duì)象還是在每個(gè)方法中使用新的連接對(duì)象,一直是很糾結(jié)的問(wèn)題,心想頻繁的數(shù)據(jù)操作還是用一個(gè)連接對(duì)象快吧?在本文給出的測(cè)試代碼中,有下列語(yǔ)句:
- //SqlConnection conn = new SqlConnection(connString);
- //conn.Open();
注釋掉這些語(yǔ)句,在被調(diào)用的方法中使用自己的連接對(duì)象,與取消注釋,全部使用一個(gè)連接對(duì)象,效率上沒(méi)有任何區(qū)別!
究其原因,可能是ADO.NET自動(dòng)使用了連接池,實(shí)際上程序在不同的情況下,使用的都是一個(gè)連接,所以操作上效率沒(méi)有區(qū)別。
后續(xù)測(cè)試
在真正的服務(wù)器上進(jìn)行測(cè)試,發(fā)現(xiàn)測(cè)試結(jié)論又不一樣,我們有服務(wù)器A,擁有16個(gè)核,32G內(nèi)存,另外一臺(tái)服務(wù)器B,擁有8個(gè)核,16G內(nèi)存。在服務(wù)器A上有一個(gè)SqlServer實(shí)例,兩個(gè)一樣的數(shù)據(jù)庫(kù);在在服務(wù)器B上有一個(gè)SqlServer實(shí)例,一個(gè)數(shù)據(jù)庫(kù),下面是測(cè)試結(jié)果:
- ------單數(shù)據(jù)庫(kù),單線程測(cè)試---------
- used all time is(ms):650
- ------同一實(shí)例,雙數(shù)據(jù)庫(kù),單線程測(cè)試---------
- used all time is(ms):418
- ------同一實(shí)例,雙數(shù)據(jù)庫(kù),多線程測(cè)試---------
- Tread 2 used all time is(ms):221
- Tread 1 used all time is(ms):223
- ====All thread completed!========
- ------雙實(shí)例,雙數(shù)據(jù)庫(kù),單線程測(cè)試---------
- used all time is(ms):1283
- ------雙實(shí)例,雙數(shù)據(jù)庫(kù),多線程測(cè)試---------
- Tread 1 used all time is(ms):228
- Tread 2 used all time is(ms):542
- ====All thread completed!========
可以看到,同一實(shí)例,多數(shù)據(jù)庫(kù),還是有明顯的優(yōu)勢(shì),而多線程優(yōu)勢(shì)更大;由于兩臺(tái)服務(wù)器性能差距較大,雙實(shí)例測(cè)試沒(méi)有顯示出優(yōu)勢(shì),但多線程下還是比單實(shí)例單數(shù)據(jù)庫(kù)好!
為什么PC機(jī)跟服務(wù)器測(cè)試的結(jié)論不一致?也許還是跟計(jì)算能力相關(guān),PC機(jī)的計(jì)算負(fù)載太大,已經(jīng)失去了測(cè)試的意義。
原文鏈接:http://www.cnblogs.com/bluedoctor/archive/2011/06/28/2092113.html
【編輯推薦】