gdb cannot attach to a running process

Март 17th, 2014 | Posted by elenst in Pensieve - (Комментарии отключены)

gdb complains on Ubuntu: “ptrace: Operation not permitted

For runtime:
echo 0 > /proc/sys/kernel/yama/ptrace_scope

Persistent:
set kernel.yama.ptrace_scope = 0 in /etc/sysctl.d/10-ptrace.conf

Obtaining and running oprofile

Февраль 20th, 2014 | Posted by elenst in Pensieve | Testing - (Комментарии отключены)

It’s not that easy to get oprofile nowadays… Shame, really.

Obtaining

For Precise, found it here: https://launchpad.net/ubuntu/+source/oprofile/0.9.6-1.3ubuntu1
Plain deb files, e.g. for amd64

https://launchpad.net/ubuntu/+source/oprofile/0.9.6-1.3ubuntu1/+build/2614177/+files/libopagent1_0.9.6-1.3ubuntu1_amd64.deb

https://launchpad.net/ubuntu/+source/oprofile/0.9.6-1.3ubuntu1/+build/2614177/+files/oprofile_0.9.6-1.3ubuntu1_amd64.deb

Amazingly,
dpkg --install libopagent1_0.9.6-1.3ubuntu1_amd64.deb oprofile_0.9.6-1.3ubuntu1_amd64.deb
worked like a charm.

For Wheezy, used the Squeeze repo:

deb http://ftp.ru.debian.org/debian squeeze main

Also had to install keys

sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com ...

and then apt-get install just worked.

Running

Installing it in the Wheezy VM was useless. It turns out that oprofile does not work in a VM.
Well, technically it works, if you set sudo modprobe oprofile timer=1, but then it returns something like that:


1043 75.8545 /data/repo/10.0/sql/mysqld my_timer_cycles
313 22.7636 /data/repo/10.0/storage/federatedx/ha_federatedx.so federatedx_io_mysql::mark_position(st_federatedx_result*, void*)
6 0.4364 /data/repo/10.0/sql/mysqld code_state
3 0.2182 /no-vmlinux /no-vmlinux
2 0.1455 /data/repo/10.0/sql/mysqld _db_enter_
...

No much use.

On Precise, it required running

sudo bash -c "echo 0 > /proc/sys/kernel/nmi_watchdog"

and then it actually worked.

To run (for mysqld):


sudo opcontrol --init
sudo opcontrol --setup --separate=lib,kernel,thread --no-vmlinux
sudo opcontrol --start-daemon
sudo opcontrol --start

Then do stuff in mysqld that needs to be profiled. Then

sudo opcontrol --dump
sudo opreport --demangle=smart --symbols --long-filenames --merge \
tgid path-to-mysqld | head -n 20
sudo opcontrol --stop
sudo opcontrol --deinit
sudo opcontrol --reset

apt and dpkg — assorted notes

Февраль 18th, 2014 | Posted by elenst in Pensieve - (Комментарии отключены)

Get detailed information about a package:
apt-cache show package-name

Find which package a file belongs to:
apt-file search filename
(might need to run apt-file update
dpkg -S filename

Find package names by a substring:
apt-cache search string

Hold a version (lock updates)
Did not check myself
apt-mark hold package-name package-name
and unhold to undo

Building sysbench 0.4

Февраль 17th, 2014 | Posted by elenst in Pensieve | Testing - (Комментарии отключены)

Download from http://netcologne.dl.sourceforge.net/project/sysbench/sysbench/0.4.12/sysbench-0.4.12.tar.gz

Get correct library and include paths by running mysql_config from the MySQL/MariaDB package that you want to use for building sysbench.

If there are two paths returned, like in MariaDB 5.5, the one where mysql.h sits seems to work well.

To link dynamically, run
./configure --with-mysql-libs= --with-mysql-includes=

To link statically, run
./configure --with-mysql-libs= --with-mysql-includes= --with-extra-ldflags=-all-static

Copy libtool from the system to the sysbench dir, otherwise it is likely to fail:

cp `which libtool` ./
ls -l ./libtool

Run

make
sysbench/sysbench --version

Building TokuDB unit tests in MariaDB tree

Февраль 16th, 2014 | Posted by elenst in MariaDB | Pensieve | Testing - (Комментарии отключены)

There are some TokuDB *.cc tests in MariaDB tree, e.g. in /storage/tokudb/ft-index/portability/tests, but they are not built by default.

Generally, to build them, we need a couple includes and one library:


g++ -c -I<basedir>/storage/tokudb/ft-index/toku_include -I<basedir>/storage/tokudb/ft-index/portability -std=c++11 test-cpu-freq.cc
g++ -o test-cpu-freq test-cpu-freq.o -L<basedir>/storage/tokudb/ft-index/portability -ltokuportability
LD_LIBRARY_PATH=<basedir>/storage/tokudb/ft-index/portability:$LD_LIBRARY_PATH ./test-cpu-freq

But on more conservative systems, there is an additional problem:

test-cpu-freq.cc: In function ‘int main()’:
test-cpu-freq.cc:103:13: error: expected ‘)’ before ‘PRIu64’

There are most certainly some smart solutions for that, but I haven’t found any that works on Wheezy, and to get it work quick, this is enough:

--- storage/tokudb/ft-index/portability/tests/test-cpu-freq.cc 2013-10-04 20:49:53 +0000
+++ storage/tokudb/ft-index/portability/tests/test-cpu-freq.cc 2014-02-16 14:35:15 +0000
@@ -100,7 +100,7 @@
int r = toku_os_get_processor_frequency(&cpuhz);
assert(r == 0);
if (verbose) {
- printf("%" PRIu64 "\n", cpuhz);
+ printf("%li\n", cpuhz);
}
assert(cpuhz>100000000);
return 0;

Also, verbose here is not an option, it’s a constant. To make the test actually verbose, modify it in the code and recompile.

Creating a local deb repository with MariaDB packages

Май 22nd, 2013 | Posted by elenst in Pensieve | Testing - (Комментарии отключены)

Based on Daniel’s email and my own experience.

Variables to set:

DEBIAN=/path/to/mkrepo-debian.sh # /path/to/mariadb-tools/buildbot/mkrepo-debian.sh)
UBUNTU="/path/to/mkrepo-ubuntu.sh" # (/path/to/mariadb-tools/buildbot/mkrepo-ubuntu.sh)
version="mariadb-10.0.2" # change to whatever makes sense
tree="10.0" # change to whatever tree you are using
tarbuildnum="3544" # change to the correct tar build number
archive_dir="/media/backup/archive/pack" # this is on hasky, but it should be local, so I need to create some $(archive_dir)/$(tree) folder locally, and copy build-$(tarbuildnum) from hasky into it.

Modify mkrepo-debian.sh and mkrepo-ubuntu.sh scripts to use my GPG key.
If needed, modify the scripts to exclude distributions that aren’t needed, especially if they are not present on hasky.

Might need to install reprepro if it’s not there yet:
apt-get install reprepro

mkdir -v /path/to/where/i/want/${version}/repo
cd /path/to/where/i/want/${version}/repo
eval $(gpg-agent --daemon)
${DEBIAN} debian ${archive_dir}/${tree}/build-${tarbuildnum}
${UBUNTU} ubuntu ${archive_dir}/${tree}/build-${tarbuildnum}

I actually didn’t run eval $(gpg-agent —daemon) , don’t even have it installed, and it still worked all right, or so it seemed.

Then, add

deb file:///path/to/where/i/want/${version}/repo/ubuntu precise main
deb-src file:///path/to/where/i/want/${version}/repo/ubuntu precise main

(and comment the previous one if there was any).
Replace ubuntu and precise with whatever you’re on.

Run

sudo apt-get update

It might complain about GPG key, then make sure it’s installed as described in http://elenst.ru/pensieve/it-bits/creating-and-installing-a-local-gpg-key-for-testing-purposes/

Then install as usual.

Creating and installing a local GPG key for testing purposes

Май 22nd, 2013 | Posted by elenst in Pensieve | Testing - (Комментарии отключены)

No fancy GUI is necessary. The base gpg is installed on Ubuntu by default.

gpg --gen-key

Use default type, size and duration.
Enter the name and the email address (I suppose it doesn’t matter which as long as it’s for testing only).
Type O to create the key.
Enter a passphrase twice.
Move your mouse like crazy, but try to spare the effort, it might take long and you’ll get tired. When it’s enough, it will tell you it’s enough. Until it does, keep moving. Don’t stop for long, or you’ll have to repeat the exercise.

The file pubring.gpg will appear in $HOME/.gnupg/

Finally,

sudo apt-key add $HOME/.gnupg/pubring.gpg

Now I can run apt-get update with repos where something is signed with this key.

RQG: SlaveCrashRecovery Reporter

Май 8th, 2013 | Posted by elenst in MariaDB | Pensieve | Testing - (Комментарии отключены)

The reporter can be used to test crash-safety of replication.

It is a periodic reporter, every 30 seconds it kills the slave server using SIGKILL, and immediately restarts it on the old data directory, with the same parameters as before. On server restart, the reporter checks that the server itself and the replication started all right.

The reporter itself does not check consistency of the data, but it can be used together with ReplicationConsistency reporter.

It is supposed to be used with runall-new.pl, so that the server is started without MTR involvement.

Some more information at https://kb.askmonty.org/en/rqg-extensions-for-mariadb-features/

RQG: LimitRowsExamined Transformer

Май 8th, 2013 | Posted by elenst in MariaDB | Pensieve | Testing - (Комментарии отключены)

There are many things in RQG for which it’s not obvious what they do by a quick look. It becomes especially embarrassing though when those are things that you developed yourself; so, I’ll try to keep track of those at least, and maybe add some records for legacy components when I can.

The LimitRowsExamined transformer checks whether the original query already contains a ROWS EXAMINED clause. If it does not, it adds the clause either after the LIMIT clause, or at the end of the query. In any case (even if ROWS EXAMINED was already there), the transformer returns the following sequence of statements:

* FLUSH STATUS
* the query with ROWS EXAMINED
* a query which sums up status variables related to examined rows

The result of the main query is checked to be a subset of the original query’s result set. The sum of status variables is checked to be not greater than the limit provided in the ROWS EXAMINED clause, plus a margin. The margin is configured in the transformer.

Some more information at https://kb.askmonty.org/en/rqg-extensions-for-mariadb-features/

MySQL ODBC connector: build and run tests

Январь 29th, 2013 | Posted by elenst in Pensieve | Testing - (Комментарии отключены)

I spent some time on it, and I have a feeling I’ll have to do it again some day, so here it goes, assuming I’ve downloaded the source package of the connector (in my case it was 5.2.3) and either installed MySQL client library, or have a binaries and headers somewhere.

I will of course need different stuff for building, and in addition to that, unixODBC and unixODBC-devel (that’s how they are called for Fedora’s yum, I didn’t check others yet).

Then, if we have MySQL client library installed system-wide, I run
cmake -G "Unix Makefiles" -DWITH_UNIXODBC=1

And if we have it in a secret place, I run
cmake -G "Unix Makefiles" -DWITH_UNIXODBC=1 -DMYSQL_INCLUDE_DIR=<secret folder with MySQL/MariaDB include files> -DMYSQL_LIB_DIR=<secret folder with MySQL/MariaDB client libs>

Either way, just hope it will find all it needs. What it needs is another story: I’ve heard today that the connector is *supposed* to link statically, only it does not. It links dynamically if it can, which kind of supported by the connectors manual. It’s rather unfortunate, since it doesn’t seem to work all that well with the current MySQL client library:

$ ldd -r lib/libmyodbc5w.so
linux-vdso.so.1 => (0x00007fffb49ff000)
libodbc.so.2 => /lib64/libodbc.so.2 (0x00007f5d1fdb8000)
libmysqlclient.so.18 => /mysql-5.5.29/lib/libmysqlclient.so.18 (0x00007f5d1f959000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f5d1f73d000)
libm.so.6 => /lib64/libm.so.6 (0x00007f5d1f442000)
libodbcinst.so.2 => /lib64/libodbcinst.so.2 (0x00007f5d1f22f000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f5d1f02b000)
libc.so.6 => /lib64/libc.so.6 (0x00007f5d1ec74000)
libltdl.so.7 => /lib64/libltdl.so.7 (0x00007f5d1ea6a000)
librt.so.1 => /lib64/librt.so.1 (0x00007f5d1e862000)
/lib64/ld-linux-x86-64.so.2 (0x000000353c200000)
undefined symbol: dynstr_append_os_quoted (lib/libmyodbc5w.so)
undefined symbol: dynstr_append (lib/libmyodbc5w.so)
undefined symbol: strfill (lib/libmyodbc5w.so)
undefined symbol: init_dynamic_string (lib/libmyodbc5w.so)
undefined symbol: dynstr_realloc (lib/libmyodbc5w.so)
undefined symbol: dynstr_free (lib/libmyodbc5w.so)
undefined symbol: dynstr_append_mem (lib/libmyodbc5w.so)

Anyway, as long as it builds, we can run at least some tests from the test folder. There is even a perl script, but it doesn’t work for me, and neither does make test. I don’t know why, and for now I don’t care. I just need to get something run.

So, I set 2 variables just like the manual says:

ODBCINI=/mysql-connector-odbc-5.2.3-src/test/odbc.ini
ODBCSYSINI=/mysql-connector-odbc-5.2.3-src/test

The second one looks stupid, but I set it anyway. Whatever it takes.

In odbc.ini, I add

[elenstdsn]
Driver=/mysql-connector-odbc-5.2.3-src/lib/libmyodbc5w.so
SERVER=localhost
UID=root
DATABASE=test
PORT=3306
Password=test

In odbcinst.ini, I add

[elenstODBC]
Driver=/mysql-connector-odbc-5.2.3-src/lib/libmyodbc5.so
UsageCount=1

Now, if I didn’t forget to start the server (importantly, on /tmp/mysql.sock, otherwise tests fail somewhere in the middle, even though they get the socket as a parameter), I can run

./test/my_basics elenstdsn root test /tmp/mysql.sock

I can’t say it works, it would be exaggeration, but at least it does something marginally sensible.