Merge branch 'master' of https://github.com/JMNetwalker/AzureSQLConnectivityChecker

Author: JMNetwalker

README.md

@@ -146,4 +146,105 @@ You need to configure the parameters:
 ### The results
 - For every interaction the results will be save on **c:\test\ResultsN.Log**, where **N** is the number of job execute.
 
+# SQLConnectivityTestMultiDriver.ps1
+This application has been designed with a main idea: We usually have service requests when our customers, using different drivers, need to know the connection latency, execution time spent or validate the connection.
+For this reason, I developed this small PowerShell script  (based on initiative of AzureSQLConnectivityChecker in multiple languages) which will basically connect to the database using 4 drivers/providers and test this.
+
+## Today, we started with SQLConnectivityTestMultiDriver.ps1 PowerShell Script and you are going to be able to:
+
+- **Connect using the drivers**: Net SqlClient, ODBC 17, OLEDB and MSOLEDBSQL.
+- **In every interaction**, you could obtain:
+  ++ Connectivity latency.
+  ++ Time invested in the execution of each query.
+  ++ Average time spent on connectivity and per query.
+  ++ How many ports the customer application has (1433,11000-12000 and others).
+  ++ Check if the DNS is resolving correct IP. 
+
+## Here are the main details:
+
+- **Parameters**, if you don't specify any of them, PowerShell Script will ask the value and check it.
+
+  + **$DatabaseServer**, will be the server name to connect. 
+  + **$Database**, will be the database name to connect.
+  + **$Username**, user name to connect
+  + **$passwordSecure**, password. 
+  + **$NumberExecutions**, how many executions that you want to run. 
+  + **$LogFile**, will be the log file that all operations will be saved. If this file is not specify, the application will ask the location. 
+  + **$Driver** to use, the values are: 
+    ++ SQLCLIENT for .NET SQL Client.
+    ++ ODBC
+    ++ MSOLEDBSQL
+    ++ SQLOLEDB
+   + $InputFile**, will be the different TSQL that you want to use. This file needs to contain per line the TSQL to execute. If you don't specify it, we are going to use SELECT 1. For example,:
+  SELECT 1
+  SELECT count(Id) FROM PerformanceVarcharnVarchar Where TextToSearch = N'Test Search ' + CONVERT(nvarchar(200),RAND()*(2000000-10000)+10000) OPTION (MAXDOP 1)
+
+## Outcome:
+In the $LogFile you are going to have all the results and summary about connection and execution spent, for example, 
+
+    2022-08-11 03:33:15 Driver: SQLOLEDB
+    2022-08-11 03:33:15 Connecting to the database: servername.database.windows.net - DB: dbname...Attempt #1 of 5 - IP:52.138.224.1
+    2022-08-11 03:33:16 Connected to the database in (ms):324.1887 - Provider:SQLOLEDB -- HostName: Server Version:12.00.0312
+    2022-08-11 03:33:16 Total Connections Failed : 0
+    2022-08-11 03:33:16 Total Connections Success: 1
+    2022-08-11 03:33:16 Total Connections ms : 324
+    2022-08-11 03:33:16 Ports Count : 1433 0 (0) 11xx: (0) Others:(215)
+    2022-08-11 03:33:16 Ports Status:
+    2022-08-11 03:33:16 ---->Established (207)
+    2022-08-11 03:33:16 ---->TimeWait (5)
+    2022-08-11 03:33:16 ---->SynSent (2)
+    2022-08-11 03:33:16 -------------------------------------------------------------------------------------
+    2022-08-11 03:33:16 Query : SELECT 1
+    2022-08-11 03:33:16 Iteration : 1 Query 1 / 2
+    2022-08-11 03:33:16 Time required (ms) : 41,416
+    2022-08-11 03:33:16 Query Commands Failed : 0
+    2022-08-11 03:33:16 Query Commands Success: 1
+    2022-08-11 03:33:16 Query Commands ms : 41
+    2022-08-11 03:33:16 Total Commands Failed : 0
+    2022-08-11 03:33:16 Total Commands Success: 1
+    2022-08-11 03:33:16 TotalCommands ms : 41
+    2022-08-11 03:33:16 -------------------------------------------------------------------------------------
+    2022-08-11 03:33:16 Query : SELECT count(Id) FROM PerformanceVarcharnVarchar Where TextToSearch = N'Test Search ' + CONVERT(nvarchar(200),RAND()*(2000000-10000)+10000) OPTION (MAXDOP 1)
+    2022-08-11 03:33:16 Iteration : 1 Query 2 / 2
+    2022-08-11 03:33:17 Time required (ms) : 473,7891
+    2022-08-11 03:33:17 Query Commands Failed : 0
+    2022-08-11 03:33:17 Query Commands Success: 1
+    2022-08-11 03:33:17 Query Commands ms : 474
+    2022-08-11 03:33:17 Total Commands Failed : 0
+    2022-08-11 03:33:17 Total Commands Success: 2
+    2022-08-11 03:33:17 TotalCommands ms : 257,5
+    2022-08-11 03:33:17 ---------------------------------------------------------------------------------------------------------------
+    2022-08-11 03:33:17 Connecting to the database: servername.database.windows.net - DB: databasena...Attempt #1 of 5 - IP:52.138.224.1
+    2022-08-11 03:33:17 Connected to the database in (ms):0 - Provider:SQLOLEDB -- HostName: Server Version:12.00.0312
+    2022-08-11 03:33:17 Total Connections Failed : 0
+    2022-08-11 03:33:17 Total Connections Success: 2
+    2022-08-11 03:33:17 Total Connections ms : 162
+    2022-08-11 03:33:17 Ports Count : 1433 0 (0) 11xx: (0) Others:(215)
+    2022-08-11 03:33:17 Ports Status:
+    2022-08-11 03:33:17 ---->Established (207)
+    2022-08-11 03:33:17 ---->TimeWait (5)
+    2022-08-11 03:33:17 ---->SynSent (2)
+
+    2022-08-11 03:38:01 -------------------------------------- SUMMARY -----------------------------------------------------------------
+    2022-08-11 03:38:01 Total Connections Failed :0
+    2022-08-11 03:38:01 Total Connections Success:200
+    2022-08-11 03:38:01 Total Connections Avg ms :2,515
+    2022-08-11 03:38:01 Total Number Executions :200
+    2022-08-11 03:38:01 Total Number Queries :2
+    2022-08-11 03:38:01 Total Number Process :400
+    2022-08-11 03:38:01 Total Commands Failed :0
+    2022-08-11 03:38:01 Total Commands Success :400
+    2022-08-11 03:38:01 Total Commands ms :120,0725
+    2022-08-11 03:38:01 ------------------- Queries SummarySELECT count(Id) FROM PerformanceVarcharnVarchar Where TextToSearch = N'Test Search ' + CONVERT(nvarchar(200),RAND()*(2000000-10000)+10000) OPTION (MAXDOP 1)
+    2022-08-11 03:38:01 Query # :SELECT 1
+    2022-08-11 03:38:01 Total Commands Failed :0
+    2022-08-11 03:38:01 Total Commands Success:200
+    2022-08-11 03:38:01 Total Commands ms :35,57
+    2022-08-11 03:38:01 Query # :SELECT count(Id) FROM PerformanceVarcharnVarchar Where TextToSearch = N'Test Search ' + CONVERT(nvarchar(200),RAND()*(2000000-10000)+10000) OPTION (MAXDOP 1)
+    2022-08-11 03:38:01 Total Commands Failed :0
+    2022-08-11 03:38:01 Total Commands Success:200
+    2022-08-11 03:38:01 Total Commands ms :204,575
+    2022-08-11 03:38:01 Review: https://docs.microsoft.com/en-us/dotnet/framework/data/adonet/sql/provider-statistics-for-sql-server
+    2022-08-11 03:38:01 -------------------------------------- SUMMARY -----------------------------------------------------------------
+
 Enjoy!

connectivityDiag.sh

@@ -3,10 +3,10 @@
 #title           :connectivityDiag.sh
 #description     :This script will launch a continuous connectivity test to databases like SQL, PostgreSQL, MySQL and MariaDB
 #authors		 :Francisco Javier Pardillo & Jose Manuel Jurado
-#date            :20201112
-#version         :0.1
+#date            :20220818
+#version         :0.2
 #usage		     :./connectivityDiag.sh
-#notes           :Install "iproute","hostname" and some of the database clients ["sqlcmd","mysql","psql"] to use this script.
+#notes           :Install "iproute","hostname","" and some of the database clients ["sqlcmd","mysql","psql"] to use this script.
 #################################################################################################################################
 LY='\033[1;33m' # Light Yellow
 LR='\033[1;31m' # Light Red
@@ -57,6 +57,8 @@ echo "Please insert NOTES of this connectivityDiag session, EJ.- TEST1 (Default:
 read NOTES
 echo "Would you like to DEBUG all commands? (Y/N) (Default=N)"
 read DEBUG
+echo "Would you like to install missing tools if necessary? (Y/N) (Default=N)"
+read INSTALL
 
 DIR=`pwd`;
 DIR_LOGS="connectivityDiag_logs";
@@ -78,18 +80,24 @@ if [ "$TEST_QUERY""a" = "a" ]; then TEST_QUERY="SELECT 1"; fi
 if [ "$TEST_WAIT""a" = "a" ]; then TEST_WAIT=1; fi
 if [ "$TEST_LOOPS""a" = "a" ]; then TEST_LOOPS=86400; fi
 if [ "$DEBUG""a" = "a" ]; then DEBUG="N"; fi
+if [ "$INSTALL""a" = "a" ]; then INSTALL="N"; fi
 
 # We need an output file per test
 logConnectivity=$DIR_LOGS/connectivity.$NOTES.$date_time.csv
 logDns=$DIR_LOGS/dns.$NOTES.$date_time.csv
-logNetstat=$DIR_LOGS/dns.$NOTES.$date_time.csv
+logNetstat=$DIR_LOGS/netstat.$NOTES.$date_time.csv
 logDebug=$DIR_LOGS/debug.$NOTES.$date_time.log
 
 # DB connections
+export PGPASSWORD=$PASSWORD
 db_postgresql="psql -h $SERVER -U $USERNAME -w$PASSWORD -d $DB -t";
 db_mysql="mysql -N -u $USERNAME -h $SERVER -p$PASSWORD -D $DB";
 db_sql="/opt/mssql-tools/bin/sqlcmd -S $SERVER -d $DB -U $USERNAME -P$PASSWORD -h-1";
 
+# Other commands
+dns="dig $SERVER +short"
+#netstat="netstat -nap|grep $PORT"
+
 CHK17="$LG""CHECK.017:Existence of directory $DIR$NC";
 CHK18="$LG""CHECK.018:Existence of directory $DIR_LOGS$NC";
 
@@ -122,9 +130,62 @@ if [ $exit_condition = "Y" ]; then echo -e "\n################################\n
 
 # DEFINE AUXILIARY FUNCTIONS 
 
+function initialSetup(){
+	out=$(which dig)
+	res=$?
+	if [ $res == 1 ];
+	then
+		
+		[[ -x "/usr/bin/apt-get" ]] && apt install -y dnsutils
+		[[ -x "/usr/bin/yum" ]] && yum install -y bind-utils
+	fi
+
+	out=$(which netstat)
+	res=$?
+	if [ $res == 1 ];
+	then
+		
+		[[ -x "/usr/bin/apt-get" ]] && apt install -y net-tools
+		[[ -x "/usr/bin/yum" ]] && yum install -y net-tools
+	fi
+
+	case $SERVERTYPE in
+    "sql")
+		out=$(which /opt/mssql-tools/bin/sqlcmd)
+		res=$?
+		if [ $res == 1 ];
+		then
+			echo -e $NC
+			echo "Missing sqlcmd command to test connectivity for SQL databases, please refer to: https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-setup-tools?view=sql-server-ver16"
+			exit 0
+		fi
+    ;;
+    "postgresql")
+		out=$(which psql)
+		res=$?
+		if [ $res == 1 ];
+		then
+			[[ -x "/usr/bin/apt-get" ]] && apt install -y postgresql-client
+			[[ -x "/usr/bin/yum" ]] && yum install -y postgresql-client
+		fi
+    ;;
+    "mysql")
+		out=$(which mysql)
+		res=$?
+		if [ $res == 1 ];
+		then
+			[[ -x "/usr/bin/apt-get" ]] && apt install -y default-mysql-client
+			[[ -x "/usr/bin/yum" ]] && yum install -y mysql
+		fi
+    ;;	
+    *)
+	#Nothing
+    ;;
+	esac
+}
 function logDebug(){
-	# $1 should be the starttime, $2 should be the output to be saved, $3 should be the endtime, $4 type of test performed
-	duration=$(($3-$1))
+	# $1 should be the starttime, $2 should be the output to be saved, $3 should be the endtime, $4 type of test performed, $5 duration
+	duration=$5
 	if [ $DEBUG == "Y" ];
 	then
 		echo "==============================" >> $logDebug
@@ -142,48 +203,63 @@ function logDebug(){
 }
 
 function chkConnMySQL(){
-	start=`date +%Y%m%d%H%M%S%N|cut -b1-17`
+	# date+Nanoseconds+secondsSince1970+Nanoseconds
+	startB=`date +%Y%m%d%H%M%S%N%s%N`
+	startC=`echo ${startB}|cut -b24-36`
+	start=`echo ${startB}|cut -b1-17`
 	out=$($db_mysql -e "$TEST_QUERY" 2>&1)
 	res=$?
 	# We need to take output in res and parse with outfin 
 	outfin=`echo $out|sed s/,/#/g|sed s/'mysql: \[Warning\] Using a password on the command line interface can be insecure\. '//g`
-	end=`date +%Y%m%d%H%M%S%N|cut -b1-17`
-	duration=$((end-start))
+	endB=`date +%Y%m%d%H%M%S%N%s%N`
+	endC=`echo ${endB}|cut -b24-36`
+	end=`echo ${endB}|cut -b1-17`
+	duration=$((endC-startC))
 	if [ $res == 0 ];
 	then
 		result='OK';
 	else
 		result='ERR';
 	fi
-	logDebug $start "$out" $end "MYSQL CONNECTIVITY"
+	logDebug $start "$out" $end "MYSQL CONNECTIVITY" $duration
 	echo $NOTES,$HOSTIPS,sqlcmd,$HOSTNAME,$start,$duration,$result,$outfin
 }
 function chkConnPostgreSQL(){
-	start=`date +%Y%m%d%H%M%S%N|cut -b1-17`
+	# date+Nanoseconds+secondsSince1970+Nanoseconds
+	startB=`date +%Y%m%d%H%M%S%N%s%N`
+	startC=`echo ${startB}|cut -b24-36`
+	start=`echo ${startB}|cut -b1-17`
 	out=$($db_postgresql -c "$TEST_QUERY" 2>&1)
 	res=$?
 	# We need to take output in res and parse with outfin 
 	outfin=`echo $out|sed s/,/#/g|sed s/'REMOVABLESTRING'//g`
-	end=`date +%Y%m%d%H%M%S%N|cut -b1-17`
-	duration=$((end-start))
+	endB=`date +%Y%m%d%H%M%S%N%s%N`
+	endC=`echo ${endB}|cut -b24-36`
+	end=`echo ${endB}|cut -b1-17`
+	duration=$((endC-startC))
 	if [ $res == 0 ];
 	then
 		result='OK';
 	else
 		result='ERR';
 	fi
-	logDebug $start "$out" $end "POSTGRESQL CONNECTIVITY"
+	logDebug $start "$out" $end "POSTGRESQL CONNECTIVITY" $duration
 	echo $NOTES,$HOSTIPS,sqlcmd,$HOSTNAME,$start,$duration,$result,$outfin
 }
 function chkConnSQL(){
-	start=`date +%Y%m%d%H%M%S%N|cut -b1-17`
+	# date+Nanoseconds+secondsSince1970+Nanoseconds
+	startB=`date +%Y%m%d%H%M%S%N%s%N`
+	startC=`echo ${startB}|cut -b24-36`
+	start=`echo ${startB}|cut -b1-17`
 	echo "SET NOCOUNT ON;SELECT $start;$TEST_QUERY"
 	out=$($db_sql -Q "SET NOCOUNT ON;SELECT $start;$TEST_QUERY" 2>&1)
 	res=$?
 	# We need to take output in res and parse with outfin 
 	outfin=`echo $out|sed s/,/\#/g|sed s/'(.*rows affected)'//g`
-	end=`date +%Y%m%d%H%M%S%N|cut -b1-17`
-	duration=$((end-start))
+	endB=`date +%Y%m%d%H%M%S%N%s%N`
+	endC=`echo ${endB}|cut -b24-36`
+	end=`echo ${endB}|cut -b1-17`
+	duration=$((endC-startC))
 	if [ $res == 0 ];
 	then
 		result='OK';
@@ -194,10 +270,65 @@ function chkConnSQL(){
 	echo $NOTES,$HOSTIPS,sqlcmd,$HOSTNAME,$start,$duration,$result,$outfin
 }
 
+function chkDns(){
+	# date+Nanoseconds+secondsSince1970+Nanoseconds
+	startB=`date +%Y%m%d%H%M%S%N%s%N`
+	startC=`echo ${startB}|cut -b24-36`
+	start=`echo ${startB}|cut -b1-17`
+	out=$($dns 2>&1)
+	res=$?
+	# We need to take output in res and parse with outfin 
+	outfin=`echo $out|sed s/,/#/g`
+	endB=`date +%Y%m%d%H%M%S%N%s%N`
+	endC=`echo ${endB}|cut -b24-36`
+	end=`echo ${endB}|cut -b1-17`
+	duration=$((endC-startC))
+	if [ $res == 0 ];
+	then
+		result='OK';
+	else
+		result='ERR';
+	fi
+	logDebug $start "$out" $end "DNS CONNECTIVITY" $duration
+	echo $NOTES,$HOSTIPS,dns,$HOSTNAME,$start,$duration,$result,$outfin
+}
+
+function chkNetstat(){
+	# date+Nanoseconds+secondsSince1970+Nanoseconds
+	startB=`date +%Y%m%d%H%M%S%N%s%N`
+	startC=`echo ${startB}|cut -b24-36`
+	start=`echo ${startB}|cut -b1-17`
+	out1="ESTABLISHED:"$(netstat -nap|grep $PORT|grep ESTABLISHED|wc -l)
+	out2="#TIME_WAIT:"$(netstat -nap|grep $PORT|grep TIME_WAIT|wc -l)
+	out3="#OTHERS:"$(netstat -nap|grep $PORT|grep -v TIME_WAIT|grep -v ESTABLISHED|wc -l)
+	out=$out1$out2$out3
+	res=$?
+	# We need to take output in res and parse with outfin 
+	outfin=`echo $out|sed s/,/#/g`
+	endB=`date +%Y%m%d%H%M%S%N%s%N`
+	endC=`echo ${endB}|cut -b24-36`
+	end=`echo ${endB}|cut -b1-17`
+	duration=$((endC-startC))
+	if [ $res == 0 ];
+	then
+		result='OK';
+	else
+		result='ERR';
+	fi
+	logDebug $start "$out" $end "NETSTAT CONNECTIVITY" $duration
+	echo $NOTES,$HOSTIPS,dns,$HOSTNAME,$start,$duration,$result,$outfin
+}
+
+if [ $INSTALL == "Y" ];
+then
+	initialSetup
+fi
 for (( l=1; l<=$TEST_LOOPS; l++ ))
 do
 	# Execute every loop because ips can change
 	HOSTIPS=$(hostname --all-ip-addresses)
+	echo `chkDns` >> $logDns
+	echo `chkNetstat` >> $logNetstat
 	case "$SERVERTYPE" in
 	"sql")  
 		echo `chkConnSQL` >> $logConnectivity