One important Android widget is AutoCompleteTextView which, as its name indicate, is a text view which allows the user to select text from among a list of possibilities based on the initial text typed:

As shown above, the user types “Ven” and is shown Venus on which the user can click on.

A second useful widget is the SeekBar which allows the user to select one value by dragging (here, a font size):

Finally, it is possible to embed a web browser within any Android application using a WebView. Note that this is different from launching a web browser activity (which launches the native Android web browser):

The work to do is to add a menu (also called an options menu) to the PermutationsReloaded application as shown in the picture. Choosing clear will erase everything on screen while choosing about will display a pop-up with some information about the app.

To create the menu, one should build one using the appropriate XML editor and override the onCreateOptionsMenu method of the activity. To handle clicks, the onOptionsItemSelected method will also need to be overriden.

Context menu

The second step is to add a context menu which opens when one long-presses on any of the permutations. The choices shown are “To uppercase” and “To lowercase” which are explicit enough.

The methods which are important here are onCreateContextMenu and onContextItemSelected as well as registerForContextMenu and unregisterForContextMenu.

Something noteworthy is that onCreateContextMenu is executed each time a context menu is shown (and not only once) and that one of its parameters is the view on which the long-press has been done.

Managing Preferences

The last enhancement to bring to the application is to allow the user to specify whether the last word entered should be remembered. If yes, the next time the application is started, the word should already appear in the EditText and the permutations shown.

To do that, a preference screen should be created using the XML editor, then a class derived from PreferenceActivity should be created and the onCreate method overriden. This new activity should be manually added to AndroidManifest.xml.

When this activity is launched (using, for example, startActivity), the Android framework will automatically store the preferences.

Finally, the application will have to be updated to store the word entered if the user has chosen to do so as well as restore that word when the application is resumed. The way to do that is to get a SharedPreference object from the PreferenceManager.

Accessing Files

Android applications can access files residing at three distinct places:

• Found in the application APK itself. This file is read-only as it is considered to be one of the static resources of the application. The work to do is to develop an application called Anagrams which given a word shows all the various anagrams of that word. Of course, this requires a dictionary and it will be stored in the APK. The key method to use here is openRawResource to get access to that dictionary.
• Found in flash memory. The second version of the application provides a simple menu (with load / save) which, depending on the item selected, either load or saves the anagrams found. This second version of the application loads and saves from flash memory.
• Found on the SD card. The third version of the application read and writes data on the SD card instead of using flash memory.

Android applications generally have four types of components:

• An activity represents a single screen with a user interface. An application can have multiple independent activities.
• A service is a component that runs in the background to perform long-running operations or to perform work for remote processes. For example, a service might play music in the background while the user is in a different application.
• A content provider manages shared application data. Other applications can query or even modify the data through the content provider.
• A broadcast receiver is a component that responds to system-wide broadcast announcements. A typical broadcast receiver might initiate a service to perform some work based on the event.

It is important to understand that any application can start another application’s component… but only indirectly as Android runs each application in a separate process. To do that, the application must deliver a message to the system that specifies its intent to start a particular component in a different application. The system then activates that component.

Work to do

Let’s start quickly. The first step is to setup the development environment as explained in the previous topic: (Java, Eclipse, Android SDK and Platform Tools).

At this point, it is possible to create an Android Virtual Device (AVD) and run an emulator.

The next step is to launch Eclipse, install (if needed) and configure the Android Development Tools plugin and start developing Android applications!

HelloWorld

The first application to develop is HelloWorld which should simply display a nice message in an activity when launched. The content of HelloWorld should be explored (AndroidManifest.xml, bin/, libs/, res/, src/, assets/, gen/ and proguard.cfg) and understood. HelloWorld should be launched in an Android emulator to check whether it’s working correctly. It is also interesting to launch it on a real Android devices (because we can) 🙂

Now

The second application to develop is called Now. It displays one button which fills in the whole of its activity. When the button is clicked, the current date and time will be updated in the button’s label.

The way to do that is to add an Android button to the activity, setOnClickListener appropriately (i.e. to update the label to the current date and time). The easiest way to do that is to use a Java inner class at this point.

The AndroidManifest.xml generated by Eclipse contains an activity, an intent-filter whose purpose is to make the activity appear in the launcher and, very importantly, specifies a minSdkVersion (which is an integer corresponding to a specific version of Android) and, optionally, a targetSdkVersion:

• 1 = Android 1.0 (obsolete)
• 2 = Android 1.1 (obsolete)
• 3 = Android 1.5
• 4 = Android 1.6 (obsolete)
• 5 = Android 2.0 (obsolete)
• 6 = Android 2.0.1 (obsolete)
• 7 = Android 2.1
• 8 = Android 2.2
• 9 = Android 2.3-2.3.2 (obsolete)
• 10 = Android 2.3.3-2.3.7
• 11 = Android 3.0
• 12 = Android 3.1
• 13 = Android 3.2
• 14 = Android 4.0-4.0.2
• 15 = Android 4.0.3-4.0.4

NowReloaded

The next application to be developed is an enhanced version of Now which, instead of using Java code to create a button uses an XML-layout generated graphically by the ADT plugin in Eclipse.

The Java code needs to be modified to use findViewById and care must be taken to really understand the difference between wrap_content and fill_parent (renamed to match_parent as from API level 8).

Forex

The next application is called Forex and allows the user to enter an amount in USD and, when the convert button is clicked, the MUR amount is calculated and displayed.

Airplanes

The application to build displays different airplanes based on which of the three radio buttons has been clicked on. The essential aspect of this application is that the radio buttons are found within a RadioGroup.

Schema optimisation

PROCEDURE ANALYSE() examines the result from a query and returns an analysis of the results that suggests optimal data types for each column that may help reduce table sizes.

It is important that the type of each column be optimal. In general, smaller is better and NULL should be avoided (especially for columns which are going to be indexed) as the NULL value imposes a performance penalty in MySQL.

The work to do is to run each table in the sample database through PROCEDURE ANALYSE() and optimise the type of the columns if required.

Indexing

MySQL supports indexes which are created with the CREATE INDEX command.

Assuming we have created an index consisting of three columns, C1, C2 and C3, the query optimiser will use this index (and therefore speed up query execution) when it matches:

• fully (i.e. C1, C2 and C3 match)
• with a leftmost prefix (C1)
• with a column prefix (beginning of C1)
• with a range of values (range of C1 values)
• with one element of C1 and a range on C2

When writing an SQL query, it is advisable to pay special attention to the WHERE clause. If an indexed column appears there, it is better for it to be placed on its own on one side of the comparison function. When this is done, the MySQL query optimiser will use the index.

The SHOW INDEX FROM command can be used to get information on existing indexes. The OPTIMIZE TABLE command is used when fragmentation occurs (either in the table or in indexes).

The work to do is to identify slow SQL queries on the sample database and decrease their runtime by judiciously using indexes.

Partitioning

MySQL can store the data in tables in distinct user-defined partitions. As written in the MySQL manual,

“In effect, different portions of a table are stored as separate tables in different locations. The user-selected rule by which the division of data is accomplished is known as a partitioning function, which in MySQL can be the modulus, simple matching against a set of ranges or value lists, an internal hashing function, or a linear hashing function.”

The effects of partitioning are diverse. First, partitioning can help in making MySQL read less data when executing a query if the query only refers to the data in a distinct partition (this is called pruning). Secondly, partitioned data is easy to maintain i.e. when having to discard old data by dropping a whole partition. Thirdly, partitions can be on different disks and therefore can be accessed concurrently.

In fact, partitioning can be thought as being a kind of coarse-grained indexing.

MySQL supports different kinds of partitions: range, list, column, etc.

Unfortunately, MySQL suffers from one very serious limitation: partitioned tables do not support foreign keys.

Partitions can be specified when doing a CREATE TABLE or, later, with ALTER TABLE.

The work to do is to add partitions to the existing tables in the same database and examine how query execution is impacted.

The MySQL event scheduler

MySQL has an event scheduler which can be used to run user-specified queries in the future and on a regular basis. By default, the scheduler is off and can be activated with SET GLOBAL event_scheduler = 1;.

When this is done, CREATE EVENT and ALTER EVENT can be used.

The work to do is to explore the various types of events which can exist and the various ways events can be created.

Benchmarking

When a database becomes big, query performance can suffer. It becomes important to measure the performance of the system and this is called benchmarking. We can measure many things like number of transactions per unit time, response time, the scalability or support for concurrency. When benchmarking, it is important not to use only a subset of real data, incorrectly distributed data or test using a single connection even though the database is supposed to be used concurrently.

ab

Various tools exist which help to benchmark a web application. One of them is the Apache Benchmarking tool (ab) which benchmarks the web application as a whole i.e. the full stack (operating system, database, web server, scripting language).

The work to do is to use ab to evaluate the performance of the web application written before.

mysqlslap

Another important benchmarking tool is mysqlslap which is distributed with the MySQL database. mysqlslap measures the performance of the database (and not the web application) and has various capabilities:

• Testing using automatically generated tables and SQL queries
• Measuring performance when the database is accessed concurrently
• Running tests many times to get more significant results
• Using existing schemas and custom SQL queries when measuring performance
• Producing performance reports

The work to do is to use mysqlslap to evaluate the performance of the standard MySQL employees sample database.

Benchmarking sometimes shows that query performance is not good enough. The question then is to know which query is too slow and to find ways to make it run more quickly. Those two steps are called profiling and optimisation.

Profiling

MySQL has a slow query log (which is inactive by default) which logs all SQL queries which are taking too long to execute. Whenever an application is slow, the slow query log should be enabled and monitored to discover the culprits. This can be done with the SET GLOBAL slow_query_log = 1; command. It is also possible to change the threshold above which queries are considered to be slow. Do SET long_query_time = 2; for 2 seconds for example.

MySQL provides many ways for a programmer (or a database administrator) to discover why an SQL query is performing badly.

MySQL server status variables provide information about the execution of queries within MySQL. There are a lot of variables but the most relevant for performance have names starting with:

• Bytes_ (bytes received and sent)
• Com_ (commands the server executed)
• Created_ (temporary tables created)
• Handler_ (storage engine operations e.g. InnoDB)
• Select_ (various types of join execution plans)
• Sort_ (various types of sort operations)

The various types of join execution plans. Some of them are:

• Select_scan: refers to a table that is completely read in sequence from the hard drive.
• Select_range: refers to a table that was read from the hard drive only in the necessary places to satisfy a limited range of conditions.
• Select_full_join: is the same as Select_scan with the difference that Select_full_join applies to the second and subsequent tables in the join plan for a multiple table query.
• etc.

The work to do is to use MySQL server status variables as well as the EXPLAIN command to fully understand how queries are executed.

(MySQL also provides the following commands:

SET profiling = 1;

SHOW PROFILES;
SHOW PROFILE FOR QUERY 1;

which give the durations of the various phases of a query)

In a lot of applications, users need to be able to upload files. For example, most social networks allow users to upload pictures.

We will modify the web application to allow users to upload pictures when they are viewing details about a specific phone. After some time, other users will be able to browse a gallery of user-uploaded pictures for each of the phones.

To do that, we will use the file upload capabilities of PHP.

It is important to make sure that the uploaded file is of the proper kind and size.

MySQL supports transactions. This allows software developers to create application which preserve data integrity even in the event of (software or hardware) failures.

The work to do is to use transactions as provided in MySQL to protect all collections of queries which should be treated as being atomic in nature.

Another important feature of MySQL is stored procedures. These procedures exist in the database and, when called from a PHP application, can greatly increase performance. Furthermore, stored procedures allow multiple applications, written in PHP or not, to behave in a consistent manner.

The work to do is simplify the checkout of cart items using stored procedures.

HTTP is stateless. Consequently:

When a web server is required to customize the content of a web page for a user, the web application may have to track the user’s progress from page to page. A common solution is the use of HTTP cookies. Another method include server side sessions.

PHP supports both sessions and cookies and, as such, allow us to transform the web application into one which supports multiple users concurrently. In other words, the work to do is to transform the web application into one where multiple users can log in and order phones at the same time. We will use PHP sessions to implement those new functionalities.

Additionally, we would like to implement a “Show popular phone of the hour”. We will use PHP cookies (with appropriate expiration dates) to implement this functionality in such a way that queries to the database are minimal.

Since version 5.0 of MySQL was released in October 2005, a lot of new features have been added to it to make it standards-compliant, secure and efficient. In 2011, MySQL 5.5 is still an open source software and is freely downloadable for various platforms. In this blog entry, I will focus on performance in MySQL.

MySQL provides two important utility statements, EXPLAIN and PROCEDURE ANALYSE, which can be used to understand where performance bottlenecks are. EXPLAIN is used to obtain information about how MySQL executes a SELECT statement. PROCEDURE ANALYSE suggests optimal data types for each column that may help reduce table sizes. Also, as MySQL exposes its inner workings to the database administrator, one can go very far in optimising performance.

One simple way to increase performance is to create indexes as they generally allow queries to run more quickly. Care must be taken to only create indexes when really needed as indexes consume space. In some special case, indexes can, in fact, decrease performance.

In the real world, web applications are used by concurrent users and it is important to use transactions and proper locking mechanisms in order to preserve the integrity of data. Of course, it is also important to understand that transactions and locking can have a negative impact on performance.

Whenever the same sequence of SQL statements is done over and over again, there is an opportunity to use a stored procedure which is just a set of SQL statements stored in the database server itself. Consequently, clients don’t need to issue the individual statements but can refer to the stored procedure instead. As written in the MySQL documentation, stored procedures can provide improved performance because less information needs to be sent between the server and the client. The tradeoff is that this does increase the load on the database server because more of the work is done on the server side and less is done on the client (application) side.

One recently added feature of MySQL is partitioning. This allows different portions of a table to be stored as separate files in different locations on disk. The rule used for doing the partitioning is specified by the user. Some queries can run much more quickly when a table is partitioned in virtue of the fact that data satisfying a given WHERE clause can be in one (or a few of the) partition(s), which obviously excludes any remaining partitions from the search.

Finally, given the prevalence of storage systems with very high I/O rates compared to our classical hard disks (e.g. solid state devices), MySQL has an adjustable I/O rate which can be tweaked to maximise performance.

We will cover some of these interesting aspects of MySQL in our coming Web Development with PHP and MySQL training starting on 25 July and 8 August.

One thing is sure: MySQL is far from being a toy.

The Linux kernel is open source software and, consequently, its source code (millions of lines of C) is available.

Sometimes, it is important (and interesting) to compile either a specific kernel module (typically because of a bug correction) or to compile the whole kernel. Of course, it is technically possible to download the latest Linux kernel, compile it and install it on an enterprise Linux distribution like RHEL or CentOS but this is not recommended.

Instead, Red Hat and CentOS recommend to obtain the source code of the kernel which is already in use in those distributions, configure it, compile it and install the compiled kernel without removing the previous one(s).

When using CentOS (for example), the first step is to obtain the source code of the kernel and unpack it correctly.

Then, depending on user requirements, the system administrator can:

• either build one kernel module (e.g. to correct a bug) and use that corrected module in the current kernel
• or rebuild the whole kernel and install it without removing existing kernels.