Author Archives: elenst

MariaDB 10.0 reference card: plugins in packages

Апрель 1st, 2014 | Posted by elenst in MariaDB - (4 Comments)

MariaDB 10.0 comes with ~50 engines and plugins; and it comes in 35 package sets (34 binary ones and a source tarball).

Every day people come asking on #maria IRC whether a package X contains an engine Y, or saying that it doesn’t, or wondering if it should. Remembering all combinations isn’t easy, and it became impractical to study build logs or package contents every time, so I ended up with a cheat sheet for 10.0.10 GA. At the very least it should help me to answer those questions; even better if somebody else finds it useful.

The tables below refer to contents of packages provided at or at MariaDB repository mirrors. Packages built by distributions might have different contents and are not covered here.


— built-in (also known as static):
the plugin comes as a part of the server binary. It can be disabled or enabled by default, but even when it is disabled, it is still known to the server and shown in SHOW ENGINES or SHOW PLUGINS with the corresponding status.

— dynamic library:
plugin is installed as an .so or .dll file in the plugin_dir. To start using it, you might need to run INSTALL SONAME '<file name>' or add plugin-load-add='<file name>' to your configuration file. If you did not do either of these, the engine/plugin will not be shown in SHOW ENGINES or SHOW PLUGINS. Currently is is the most common reason why people complain about “missing” engines.

Please note that you can run
to see available plugins.

— separate package: plugin is provided as a separate .rpm or .deb package in the repository along with server and client packages. To start using it, you need to install the package first (it will put the library into the plugin folder), and then possibly enable it as a usual dynamic library.


Some engines are built-in and enabled in all binary packages:

— Aria
— Memory
— Merge
— XtraDB (default engine)
— Performance schema

The rest can vary from package to package. The summary is below.

Single-file packages (bintar, zip, msi)
Generic bintar GLIBC_2.14 bintar Windows zip Windows msi
x86 amd64 x86 amd64 x86 amd64 x86 amd64
Archive built-in .dll
Blackhole built-in .dll
Connect .so .dll
Example .so .dll
Federated .dll
FederatedX built-in .dll
InnoDB .so .dll
Oqgraph .so
Sequence .so .dll
Sphinx .so .dll
Spider .so .dll
Test SQL Discovery .so .dll
TokuDB .so
x86 amd64 x86 amd64 x86 amd64 x86 amd64
Generic bintar GLIBC_2.14 bintar Windows zip Windows msi


It might happen that an engine provided as a dynamic library in a bintar is not installable on some systems. he reason is that it is not feasible to have a bintar for each OS flavor. For example, Connect engine from Generic bintar will not be installable on Centos 6, because it requires, while Centos 6 has TIn such cases, OS-specific packages should be used instead.

The absence of Cassandra engine in GLIBC_2.14+ bintar should be resolved later.

Deb packages
Squeeze Wheezy Sid Lucid Precise Quantal Saucy Trusty
x86 amd64 x86 amd64 x86 amd64 x86 amd64 x86 amd64 x86 amd64 x86 amd64 x86 amd64
Archive built-in
Blackhole built-in
Cassandra .so .so .so
Connect separate package
FederatedX built-in
InnoDB .so
Oqgraph separate package
Sequence .so
Sphinx .so
Spider .so
Test SQL Discovery .so (in test package)
TokuDB .so .so .so .so .so
x86 amd64 x86 amd64 x86 amd64 x86 amd64 x86 amd64 x86 amd64 x86 amd64 x86 amd64
Squeeze Wheezy Sid Lucid Precise Quantal Saucy Trusty

Note: The absence of Cassandra engine for Quantal (and probably for Sid) should be resolved later.

RPM packages
Centos5 RHEL5 Centos6 Fedora19 Fedora20
x86 amd64 x86 amd64 x86 amd64 x86 amd64 x86 amd64
Archive built-in
Blackhole built-in
Cassandra separate package
Connect separate package
Example .so (in test package)
FederatedX built-in
InnoDB .so
Oqgraph separate package
Sequence .so
Sphinx .so
Spider .so
Test SQL Discovery .so (in test package)
TokuDB .so .so .so
x86 amd64 x86 amd64 x86 amd64 x86 amd64 x86 amd64
Centos5 RHEL5 Centos6 Fedora19 Fedora20

Other plugins

Situation with other plugins is less complicated and rarely causes questions.

There are, again, several plugins which are built-in in all binary packages:

— Binlog (pseudo SE to represent the binlog in a transaction)
— User feedback plugin (disabled by default)
— Native MySQL authentication
— Old MySQL-4.0 authentication
— Partition SE helper (formally it is an engine, but I find it more appropriate to put it here)

Some plugins are present as dynamic libraries in all packages:

— Locale list
— Metadata lock info
— Query cache info
— Query response time
— Semisync master
— Semisync slave
— Server audit
— SQL error log

Platform-specific plugins:

— HandlerSocket (in all Linux packages)
— PAM authentication plugin (in all Linux packages)
— Unix socket authentication plugin (in all Linux packages)

Client plugins present everywhere as dynamic libraries (in bintars, zips, msi, ‘libmariadbclient18′ debs, ‘shared’ RPMs)

— Dialog
— Clear password auth plugin
— Windows authentication plugin (in all Windows packages)

Test plugins (in bintars, zips, ‘test’ debs and RPMs):

— NULL audit
— Test for API 0x0100 support (old API support)
— Dialog plugin demos
— Daemon example
— Full-text parser
— Plugin API tests

gdb cannot attach to a running process

Март 17th, 2014 | Posted by elenst in Pensieve - (Комментарии отключены)

gdb complains on Ubuntu: “ptrace: Operation not permitted

For runtime:
echo 0 > /proc/sys/kernel/yama/ptrace_scope

set kernel.yama.ptrace_scope = 0 in /etc/sysctl.d/10-ptrace.conf

External bug reports #2: Build your portfolio

Март 3rd, 2014 | Posted by elenst in MariaDB | Testing - (Комментарии отключены)

While user bug reports are the most important ones, there is a category of external reporters which I historically have a special interest in and great expectations for: entry-level testers. I was one, trained some, interviewed many, had a few hired, and have always wanted someone to wake them up and get going before it’s too late.

There is no secret that quality control is not as glamourous as other IT specialities, and there are no famous (or maybe any) student programs for testers. Usually people come into testing because it is deceptively open for newbies, planning to obtain a few points for a CV and switch either to development or to project management as soon as they can. Most do, a few stay.

It creates a vicious cycle. Since this is a well-known pattern, employers hire testers without experience not to teach them, but to let them do brainless mundane work. In turn, newcomers gain a distorted impression of the speciality, get discouraged and move on long before the job has a chance to grow on them.

But it doesn’t have to be like that. Nowadays a new-born tester can get good karma before starting the first job, and enter the market as an equal. The job is very different when you are hired as a specialist and not as a cog, the work won’t be nearly as frustrating, and you’ll be able to make a well thought-out decision about the future.

First CV: skills and portfolio

Since you don’t have any work experience to put on the CV, you usually have to go with education and skills. You can add another important part though — a bug report portfolio.

Particular technical skills are mainly important for the initial screening, when some HR simply match your CV against a wishlist that they have. You still need to gain some to pass, since you mustn’t lie in your resume, ever. But once you get through to the technical interviewers, if they know the first thing about testing, they will look at two main points:

  • whether you know what it means to be a tester;
  • whether you show any potential for being a good one.

Testing is not easy, and your interviewer is painfully aware of that, so it’s important to show that you have already tried, somewhat succeeded, and made a conscious decision to keep doing it.

That’s something you can prove by providing the bug report portfolio.

It reveals a lot about you, much more than just your hunt-fu. While a CV is only something you can say about yourself, your public portfolio shows what the world can say about you as a tester, since your bug reports are evaluated by independent parties. Only a really dumb interviewer could ignore it.

So, it is worthwile to make sure you create a good one.

How to choose the project(s)?

The common idea is simple: you choose a project with a public bug tracker and start testing it.

If you want to deal with the source code, obviously the project has to be open-source.

If there is a particular field you want to get familiar with, choose a project where you can focus on it. There is no need to worry about testing technologies: you can use whichever you like, or want to learn, and apply to the product of your choice.

Also, the product should be fairly known, so that the interviewer can easily understand the context of reports.

Main traits of a bug report portfolio

An interviewer will try to quickly evaluate your portfolio. Thus, it’s important that all bugs can be viewed, searched and filtered by a non-logged user, and that you can create a permanent link to a list of your reports only, preferably with some sorting.

The list should display a summary of the report, its severity/priority from the project’s point of view, status/resolution.
It should either contain all your reports, or come with an explanation which filters were applied, and how to remove them. Don’t hide “bad” reports behind obscure filters. Instead, make sure that most of your reports are really good.

Severity/priority shows your hunting skill. It is fine to report minor issues and cosmetic problems if you notice them, but if most of your reports are like that, it just won’t impress the interviewer. One killed bear is worth a dozen of quail… Although, be smart about it. People like juicy crashes, but sometimes non-crashing functional bugs are much more important, and hopefully your future interviewers know that.

Status/resolution shows a lot more.

The more fixed bugs you have on your list, the better — it means the reports were valid, and were taken seriously. Confirmed but not fixed bugs are not bad either.

Many “Can’t reproduce” bugs is a very bad sign — it means you either cannot provide reproducible test cases, or cannot communicate them to the project people. Both are crucial for a good tester.

Many “Duplicate” bugs is also bad — it means you don’t bother or cannot search the bug base before filing a bug. Nobody likes those who don’t use the search.

Many “Not a bug” entries is suspicious — it looks like you don’t check documentation with due diligence.

Many bugs which were just submitted but didn’t get any feedback don’t say anything about you as a tester, only show that you probably chose a wrong project to work with.

Watch for common flaws in bug tracking routine

Disclaimer: Any resemblance to bug trackers living or dead should be plainly apparent to those who know them, especially because the author has been grumbling about them for really long time…

The criteria above are reasonable in theory. Unfortunately, it happens that you get your portfolio spoiled without any fault from your side.

To display the real ratio of your report severity, the bug base should provide access to all reports. If the critical bugs remain hidden, then the visible list will only contain mediocre bugs, and the better your real list was, the less impressive your portfolio will look.

It often happens that a report is initially confirmed, stays in the queue for a (long) while, and eventually gets closed because the bug is no longer reproducible in the current version.
Not only is it unfair towards the reporter, but is also dangerous for the project. The project’s problem is another story, but for you it means that you’ll have an unfair share of bad “Can’t reproduce” reports on the list.

Wrong “Duplicate” status comes in different flavours.
First, it can be a consequence of hidden bugs. If something is not searchable, you cannot find it, so you reasonably file a new one, and it gets closed as a duplicate.
Secondly, the problem can be similar to “Can’t reproduce” ones. You file a bug in June, it is confirmed and stays in the queue, then 3 months later somebody else reports the same issue, it gets fixed, and the older bug is marked a duplicate. There is no harm for the product, so developers find this unimportant (but try to call them plagiarists after somebody copied their code, and see if you live through the day…).
Another, more subtle issue with “Duplicates” is that it can be set after a deep analysis by a developer, because, although bugs look nothing alike, they (allegedly) have the same root cause. Again, doing so is wrong for the reporter and dangerous for the product.

Also, numerous “Not a bug” problem can be caused by a lack of documentation. If you can’t find out how something is supposed to work, you are likely to waste time on reporting something that will be dismissed. So, make sure that the product is documented.

Most of these problems do not really concern users or senior testers, just maybe irritate them. For you, they are critical. While you’ll have very good excuses, keep in mind that you won’t be around when the interviewer initially evaluates your list. So, try to avoid it in advance — browse the tracker of your choice, check the history of some reports, see how things are done there. Exceptions can and do happen, but they should remain rare.

Why MariaDB?

Of course, my goal is to convince new testers not only to start testing, but to choose MariaDB over other projects to perfect their skills on. While I’m somewhat biased, I still think that MariaDB is objectively a good choice:

  • It is a crazy open-source mix of various technologies — numerous engines, plugins, connectors, replication, Galera, optimizer… Whatever you want to focus on, unless it is seriously exotic, you will most likely find something related at MariaDB;
  • MariaDB Knowledge Base contains a lot of information about the products; also, for the most part the MySQL manual applies;
  • All our bug reports are public, so the visibility problems do not exist;
  • JIRA which we use for bug reporting is hardly the best system in the world, but it does provide enough searching/filtering to fulfill all the needs associated with building a portfolio;
  • We release early, and unstable early releases are golden mine for eager testers;

And last but not least — it takes one to know one; you can expect that your interests will be taken seriously. We cut some corners when it comes to internal reports, but I’ve been trying to make sure that our external reporters are treated fairly, and I think our developers started picking up the idea. So, if something was done wrong with your report, you can let us know without a fear of being laughed at, and we’ll try to make it right if the request is reasonable.


If you indicate that you are a tester, I can provide you with some hints on how to improve your report, or some other community members might (we don’t bother users with improving reports as long as there is enough for us to go on). If there is a demand, most interesting cases can be later analyzed in blog posts; finally, if you are a regular reporter, and a really good one, we can also provide you with a reference for your first CV.

How to start

It is really this simple.

External bug reports #1: Last call for early adopters

Март 1st, 2014 | Posted by elenst in MariaDB - (Комментарии отключены)

MariaDB 10.0.9, tentatively the last RC, is almost out. MariaDB 10.0.10 GA is coming shortly after.

I envision all those who is planning to upgrade to 10.0 as soon as it becomes stable, waiting for the GA to throw a switch.
I know the feeling. As a user, I never take anything too new either.
But you are not just users, right? If you are in charge of upgrade, you are superusers, you are smart and cautious, you won’t put it in production just like that, but will try in a staging first. So do it, do it now, it’s already time. You won’t gain much by waiting further; but you’ll most certainly lose something.

Of course you have every right to expect a stable reliable version once it’s called GA, and to be irritated if you encounter a problem, and to blame us for that. But it won’t help you get what you need.
See, there are always bugs, GA or not GA. Most of them won’t affect you, but some might. Nobody can evaluate this soon-to-be-GA product in regard to your tasks and goals better than you.

There won’t be huge changes between the RC and GA. Install now, try, report while it is still RC — and there is a good chance you’ll get the fix in time, and possibly even a better one than you’d have if you reported after-GA: as everyone knows, the further after GA, the less intrusive (and sometimes less efficient) bugfixes become. Also, keep in mind that we always give higher priority to external bug reports comparing to those of comparable severity, but filed from inside.

Why do I care whether you install or not?

There is the everlasting argument whether it’s right or wrong to release early and let the community try out the product before it ripens. Ironically, quality control people are often the most fierce antagonists. Why, is it not a good thing when somebody helps you “do your job”?

Well, most testers are still marginally humans. While we know better than anybody that there are always bugs, it’s still excruciating when somebody finds one after you; and much more so if this somebody is an external user. Besides, for many testers the cool part of the job is hunting and finding fancy crashes, while working with external reports is a mundane routine.

So, all in all, we begin to rationalize. The argument that QC often comes up with is that external bugs are mostly a useless waste of time, they are unimportant, badly reported, and so on. When higher authorities are involved, the rationale sounds more elegant, something like not wishing to burden the community with doing the dirty work. But underneath, it’s all the same, trying to keep appearances.

I think at the end it works against the quality of the product.

Maybe I’m just lucky, or maybe MariaDB users are the best, but I’m getting really great bug reports from the community all the time. Some are nearly perfect right away, many require additional work and clarification, and yes, it takes time to process them, and — oh yes — I feel very bad about every bug that we hadn’t caught before the users did; but anyway, this contribution is invaluable, because it represents the real users’ problems, and because, no matter how hard we work on testing the product, we can’t compete with the whole universe of users who have all imaginable (and often unimaginable) kinds of workflow, data, environments, combinations of the third-party software, you name it.

So, please install, try, report, and thanks to everyone who is already doing that.

Obtaining and running oprofile

Февраль 20th, 2014 | Posted by elenst in Pensieve | Testing - (Комментарии отключены)

It’s not that easy to get oprofile nowadays… Shame, really.


For Precise, found it here:
Plain deb files, e.g. for amd64

dpkg --install libopagent1_0.9.6-1.3ubuntu1_amd64.deb oprofile_0.9.6-1.3ubuntu1_amd64.deb
worked like a charm.

For Wheezy, used the Squeeze repo:

deb squeeze main

Also had to install keys

sudo apt-key adv --recv-keys --keyserver ...

and then apt-get install just worked.


Installing it in the Wheezy VM was useless. It turns out that oprofile does not work in a VM.
Well, technically it works, if you set sudo modprobe oprofile timer=1, but then it returns something like that:

1043 75.8545 /data/repo/10.0/sql/mysqld my_timer_cycles
313 22.7636 /data/repo/10.0/storage/federatedx/ federatedx_io_mysql::mark_position(st_federatedx_result*, void*)
6 0.4364 /data/repo/10.0/sql/mysqld code_state
3 0.2182 /no-vmlinux /no-vmlinux
2 0.1455 /data/repo/10.0/sql/mysqld _db_enter_

No much use.

On Precise, it required running

sudo bash -c "echo 0 > /proc/sys/kernel/nmi_watchdog"

and then it actually worked.

To run (for mysqld):

sudo opcontrol --init
sudo opcontrol --setup --separate=lib,kernel,thread --no-vmlinux
sudo opcontrol --start-daemon
sudo opcontrol --start

Then do stuff in mysqld that needs to be profiled. Then

sudo opcontrol --dump
sudo opreport --demangle=smart --symbols --long-filenames --merge \
tgid path-to-mysqld | head -n 20
sudo opcontrol --stop
sudo opcontrol --deinit
sudo opcontrol --reset

apt and dpkg — assorted notes

Февраль 18th, 2014 | Posted by elenst in Pensieve - (Комментарии отключены)

Get detailed information about a package:
apt-cache show package-name

Find which package a file belongs to:
apt-file search filename
(might need to run apt-file update
dpkg -S filename

Find package names by a substring:
apt-cache search string

Hold a version (lock updates)
Did not check myself
apt-mark hold package-name package-name
and unhold to undo

Building sysbench 0.4

Февраль 17th, 2014 | Posted by elenst in Pensieve | Testing - (Комментарии отключены)

Download from

Get correct library and include paths by running mysql_config from the MySQL/MariaDB package that you want to use for building sysbench.

If there are two paths returned, like in MariaDB 5.5, the one where mysql.h sits seems to work well.

To link dynamically, run
./configure --with-mysql-libs= --with-mysql-includes=

To link statically, run
./configure --with-mysql-libs= --with-mysql-includes= --with-extra-ldflags=-all-static

Copy libtool from the system to the sysbench dir, otherwise it is likely to fail:

cp `which libtool` ./
ls -l ./libtool


sysbench/sysbench --version

Building TokuDB unit tests in MariaDB tree

Февраль 16th, 2014 | Posted by elenst in MariaDB | Pensieve | Testing - (Комментарии отключены)

There are some TokuDB *.cc tests in MariaDB tree, e.g. in /storage/tokudb/ft-index/portability/tests, but they are not built by default.

Generally, to build them, we need a couple includes and one library:

g++ -c -I<basedir>/storage/tokudb/ft-index/toku_include -I<basedir>/storage/tokudb/ft-index/portability -std=c++11
g++ -o test-cpu-freq test-cpu-freq.o -L<basedir>/storage/tokudb/ft-index/portability -ltokuportability
LD_LIBRARY_PATH=<basedir>/storage/tokudb/ft-index/portability:$LD_LIBRARY_PATH ./test-cpu-freq

But on more conservative systems, there is an additional problem: In function ‘int main()’: error: expected ‘)’ before ‘PRIu64’

There are most certainly some smart solutions for that, but I haven’t found any that works on Wheezy, and to get it work quick, this is enough:

--- storage/tokudb/ft-index/portability/tests/ 2013-10-04 20:49:53 +0000
+++ storage/tokudb/ft-index/portability/tests/ 2014-02-16 14:35:15 +0000
@@ -100,7 +100,7 @@
int r = toku_os_get_processor_frequency(&cpuhz);
assert(r == 0);
if (verbose) {
- printf("%" PRIu64 "\n", cpuhz);
+ printf("%li\n", cpuhz);
return 0;

Also, verbose here is not an option, it’s a constant. To make the test actually verbose, modify it in the code and recompile.

Creating a local deb repository with MariaDB packages

Май 22nd, 2013 | Posted by elenst in Pensieve | Testing - (Комментарии отключены)

Based on Daniel’s email and my own experience.

Variables to set:

DEBIAN=/path/to/ # /path/to/mariadb-tools/buildbot/
UBUNTU="/path/to/" # (/path/to/mariadb-tools/buildbot/
version="mariadb-10.0.2" # change to whatever makes sense
tree="10.0" # change to whatever tree you are using
tarbuildnum="3544" # change to the correct tar build number
archive_dir="/media/backup/archive/pack" # this is on hasky, but it should be local, so I need to create some $(archive_dir)/$(tree) folder locally, and copy build-$(tarbuildnum) from hasky into it.

Modify and scripts to use my GPG key.
If needed, modify the scripts to exclude distributions that aren’t needed, especially if they are not present on hasky.

Might need to install reprepro if it’s not there yet:
apt-get install reprepro

mkdir -v /path/to/where/i/want/${version}/repo
cd /path/to/where/i/want/${version}/repo
eval $(gpg-agent --daemon)
${DEBIAN} debian ${archive_dir}/${tree}/build-${tarbuildnum}
${UBUNTU} ubuntu ${archive_dir}/${tree}/build-${tarbuildnum}

I actually didn’t run eval $(gpg-agent —daemon) , don’t even have it installed, and it still worked all right, or so it seemed.

Then, add

deb file:///path/to/where/i/want/${version}/repo/ubuntu precise main
deb-src file:///path/to/where/i/want/${version}/repo/ubuntu precise main

(and comment the previous one if there was any).
Replace ubuntu and precise with whatever you’re on.


sudo apt-get update

It might complain about GPG key, then make sure it’s installed as described in

Then install as usual.

Creating and installing a local GPG key for testing purposes

Май 22nd, 2013 | Posted by elenst in Pensieve | Testing - (Комментарии отключены)

No fancy GUI is necessary. The base gpg is installed on Ubuntu by default.

gpg --gen-key

Use default type, size and duration.
Enter the name and the email address (I suppose it doesn’t matter which as long as it’s for testing only).
Type O to create the key.
Enter a passphrase twice.
Move your mouse like crazy, but try to spare the effort, it might take long and you’ll get tired. When it’s enough, it will tell you it’s enough. Until it does, keep moving. Don’t stop for long, or you’ll have to repeat the exercise.

The file pubring.gpg will appear in $HOME/.gnupg/


sudo apt-key add $HOME/.gnupg/pubring.gpg

Now I can run apt-get update with repos where something is signed with this key.