Spring Framework and uber-jar made by maven-shade-plugin

If you’re trying to build a Java application in a form of an uber-jar with use of Spring Framework, maven and maven-shade-plugin and you see an error similar to the below one (BeanDefinitionParsingException: Configuration problem: Unable to locate Spring NamespaceHandler for XML schema namespace) after running the application then this article is for you.

org.springframework.beans.factory.parsing.BeanDefinitionParsingException: Configuration problem: Unable to locate Spring NamespaceHandler for XML schema namespace [http://www.springframework.org/schema/context]
Offending resource: class path resource [app-context.xml]

	at org.springframework.beans.factory.parsing.FailFastProblemReporter.error(FailFastProblemReporter.java:70)
	at org.springframework.beans.factory.parsing.ReaderContext.error(ReaderContext.java:118)
	at org.springframework.beans.factory.parsing.ReaderContext.error(ReaderContext.java:110)
	at org.springframework.beans.factory.xml.BeanDefinitionParserDelegate.error(BeanDefinitionParserDelegate.java:301)
	at org.springframework.beans.factory.xml.BeanDefinitionParserDelegate.parseCustomElement(BeanDefinitionParserDelegate.java:1407)
	at org.springframework.beans.factory.xml.BeanDefinitionParserDelegate.parseCustomElement(BeanDefinitionParserDelegate.java:1400)
	at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.parseBeanDefinitions(DefaultBeanDefinitionDocumentReader.java:172)
	at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.doRegisterBeanDefinitions(DefaultBeanDefinitionDocumentReader.java:142)
	at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.registerBeanDefinitions(DefaultBeanDefinitionDocumentReader.java:94)
	at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.registerBeanDefinitions(XmlBeanDefinitionReader.java:508)
	at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:392)
	at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:336)
	at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:304)
	at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:181)
	at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:217)
	at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:188)
	at org.springframework.web.context.support.XmlWebApplicationContext.loadBeanDefinitions(XmlWebApplicationContext.java:125)
	at org.springframework.web.context.support.XmlWebApplicationContext.loadBeanDefinitions(XmlWebApplicationContext.java:94)
	at org.springframework.context.support.AbstractRefreshableApplicationContext.refreshBeanFactory(AbstractRefreshableApplicationContext.java:129)
	at org.springframework.context.support.AbstractApplicationContext.obtainFreshBeanFactory(AbstractApplicationContext.java:614)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:515)
	...

Solution

The solution is to extend maven-shade-plugin configuration in your pom.xml as given in documentation for maven-shade-plugin Resource Transformers. The important part to add is:

<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
    <resource>META-INF/spring.handlers</resource>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
    <resource>META-INF/spring.schemas</resource>
</transformer>

The complete configuration of maven-shade-plugin is therefore something like this:

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-shade-plugin</artifactId>
	<version>3.1.0</version>
	<executions>
	    <execution>
		<phase>package</phase>
		<goals>
		    <goal>shade</goal>
		</goals>
		<configuration>
		    <createDependencyReducedPom>false</createDependencyReducedPom>
		    <transformers>
		        <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
		            <mainClass>fully.qualified.your.main.class</mainClass>
		        </transformer>
		        <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
		            <resource>META-INF/spring.handlers</resource>
		        </transformer>
		        <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
		            <resource>META-INF/spring.schemas</resource>
		        </transformer>
		    </transformers>
		    <filters>
		        <filter>
		            <artifact>*:*</artifact>
		            <excludes>
		                <exclude>META-INF/*.SF</exclude>
		                <exclude>META-INF/*.DSA</exclude>
		                <exclude>META-INF/*.RSA</exclude>
		            </excludes>
		        </filter>
		    </filters>
		</configuration>
	    </execution>
	</executions>
</plugin>

The above worked for me. Parameters <filters> and <createDependencyReducedPom/> were accompanying some sample plugin configuration I found and they’re not breaking things in my case.

Advertisements
Posted in Java, Spring | Tagged , , | Leave a comment

How to install Skype 8 on Linux (Ubuntu & Mint)

An installation of Skype in version 8.xx is a little bit different than it was with Skype 4.xx from what I found. As of writing this the current version of Skype is 8.11 and the procedure that I’ve checked on 2 computers was:

  1. Download DEB package with Skype from https://www.skype.com
  2. Check if GDebi is installed in your system. If not, install it from a repository. I used Synaptic package manager for this.
  3. Open downloaded DEB file (skypeforlinux-64.deb) with GDebi. To to this right-click on the file. On Linux Mint with Mate the context menu already contained “Open with GDebi” as the first menu-item. On Linux Ubuntu with Cinnamon one has to select “Open with…” menu-item and then select GDebi from the sub-menu.
  4. Click on “Install” button and provide your password

Done!

This worked for me under Linux Mint 17.3 and Linux Ubuntu 16.04 both being 64-bit.

Afterword

Please note that Skype is not an open-source software and I don’t recommend it. This article is just providing an instruction how to get latest Skype version working on Linux in case you need it.

If you know the open-source alternative to Skype that is available on Linux and allows video-calls and it quite stable and somehow popular then please let me know.

Posted in Linux | 2 Comments

Gradle vs Maven

Short story

Don’t use gradle. Use maven.

A little longer story

Don’t use gradle until you really have to. Otherwise use maven.

Long story

Last time I was dropped into a project that was using gradle. And it was the first time during my professional career in Java (11 years!) I’ve encountered gradle in the wild. Moreover in the team there were 3 younger developers having different degree of experience. None of them knew gradle as well and preferred to use maven. Some 3 years before this in another company at the start of a new project we were discussing what build tool to use. Around 20 people and only one of them proposed to use gradle. I think I don’t have to say we’ve decided to use maven.

Conclusion 1: Maven is much more popular and gradle is just exotic. Maven is familiar to most Java developers so choosing gradle will make it harder to maintain the project.

Maven build files (pom.xml) are expressed in XML. XML is a kind of a document so maven build files are documents. It’s good. On the other hand gradle build files (build.gradle) are expressed in something similar to Groovy programming language. That’s make them programs! And as usual in a computer program you can do things in many different ways. And Groovy is even making things worse! There are many optional syntax elements – so they can appear or not (like semicolons – please refer to DZone: Groovy, Sometimes You Still Need a Semicolon.).

Conclusion 2: Gradle build files can be much more messy than Maven build files. Moreover Maven build files are documents you just read while Gradle build files are programs you’re wondering what they do.

Moreover my personal impression was that building a project with gradle is slower than building with maven. It seems others experienced this as well: How to speed up your slow Gradle builds. It’s crazy to use this tool until you really have to!

Posted in Java | Tagged , | 2 Comments

Network problem inside a Docker container

Last weeks me and couple of folks from my team have started using Docker. So every one of us was a Docker-newbie. And I have noticed all of us had the same common problem:

Network connections were not working inside a Docker while they were fine on a host running the Docker container.

Example: I wanted to build a Docker image based on the following Dockerfile:

FROM debian:9.2
RUN apt-get update
RUN apt-get install -y openjdk-8-jdk
RUN apt-get install -y maven
RUN apt-get install -y gradle

My command to build an image:
docker build -t java8_maven_gradle .

It failed as soon as it started executing the apt-get update command. None of other commands making network requests was running as well.

After some surprisingly long search in the Internet I found out that a Docker container by default has networking turned off. That explained our problems. The solution was to add
--network=host
parameter to commands invoking Docker. So my command became:
docker build --network=host -t java8_maven_gradle .

I used version 17.xx of Docker. Some older versions of Docker do not recognize --network parameter. Instead they need -net.

Afterwords

I suppose this is something obvious for people knowing Docker well. However it looks like people that need to just quickly start using Docker without devoting a longer time for studying are facing this problem often. I guess this is something Docker tutorial/beginners guides don’t cover good enough.

Posted in Uncategorized | Tagged , | 1 Comment

Spring Boot configuration for Tomcat’s pooling data source

In one of projects I was just working on in my professional career I’ve faced the problem of a Spring Boot configuration for a Tomcat’s database connection pool. The main issue was that after some time the pool was giving database connections that were not valid so we were trying different pool configuration parameters – but this is not relevant here. What is relevant is that all re-configuring approaches had no effect.

Is Spring Boot properly configuring the data source?

At some point I stopped believing that the Spring Boot is applying data source parameters from configuration file which was application.yml having following part related to the data source:

spring:
  datasource:
    url: jdbc:mysql://localhost:3306/some_db
    username: root
    password: root
    driver-class-name: "com.mysql.jdbc.Driver"
    tomcat:
      removeAbandoned: true
      removeAbandonedTimeout: 120
      logAbandoned: true
      maxActive: 50
      maxIdle: 5
      maxWait: 1000
      validationQuery: "select 1"
      testOnBorrow: true
      testOnConnect: true
      testWhileIdle: true

I’ve realized that I cannot easily verify what are actual data source parameters being applied by the Spring Boot. At last I’ve added a diagnostic REST endpoint showing what are actual parameters of the data source configured by Spring Boot. It was looking like this:

import org.apache.commons.beanutils.BeanUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;
import javax.sql.DataSource;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.TreeMap;

@RestController
public class InfoController {
    private static final Logger LOGGER = LoggerFactory.getLogger(InfoController.class);
    private static final List<String> TOMCAT_DB_CONN_POOL_PROPERTIES = Arrays.asList(
            "driverClassName", "maxActive", "maxAge", "maxIdle", "maxWait", "minEvictableIdleTimeMillis",
            "minIdle", "name", "removeAbandonedTimeout", "timeBetweenEvictionRunsMillis", "validationInterval",
            "validationQuery", "validationQueryTimeout", "removeAbandoned", "testOnBorrow", "testWhileIdle",
            "testOnConnect", "poolSweeperEnabled", "logAbandoned", "jmxEnabled", "url", "dataSourceJNDI",
            "initialSize", "suspectTimeout", "username", "useDisposableConnectionFacade");

    @Autowired
    private DataSource dataSource;

    @RequestMapping(path = "/data-source-info", method = RequestMethod.GET, produces = "application/json")
    public Object getInfo() {
        Map<String, Object> dataSourceDetails = new TreeMap<>();
        dataSourceDetails.put("data-source-class", dataSource.getClass().getName());
        addObjPropertiesUnderKey(dataSource, "data-source-config", dataSourceDetails);
        return dataSourceDetails;
    }

    static void addObjPropertiesUnderKey(Object obj, String key, Map<String, Object> map) {
        try {
            Map<String, Object> properties = new TreeMap<>();
            TOMCAT_DB_CONN_POOL_PROPERTIES.stream().forEach(property -> addPropertyIfNotNull(obj, property, properties));
            if (!properties.isEmpty()) {
                map.put(key, properties);
            }
        } catch (Exception e) {
            LOGGER.warn("Cannot fetch properties of object: {}.", obj, e);
        }
    }

    static void addPropertyIfNotNull(Object obj, String property, Map<String, Object> map) {
        try {
            String value = BeanUtils.getProperty(obj, property);
            if (value != null) {
                map.put(property, value);
            }
        } catch (Exception e) {
            LOGGER.warn("Cannot get property '{}' from  object of class {}. {}", property, obj.getClass(), e.getMessage());
        }
    }
}

The above solution uses dynamic property access with BeanUtils to not bind the application code to the actual data source implementation class (org.apache.tomcat.jdbc.pool.DataSource in case of Tomcat db-connection pool). And of course the set of property names is specific for this data source class.

The first look at the output produced by this REST controller confirmed my suspicion. Only first 4 parameters were properly applied (url, user, password and driver class). The remaining parameters (all with prefix spring.datasource.tomcat.) were ignored by the Spring Boot!

Spring Boot versions and the parameters naming

It turned out the data source parameters definition given in project’s application.yml file was conforming to the current version of the Spring Boot – which was 1.5.8 at the time of writing this article. But the Spring Boot version used in the project was much older: 1.3.8.

Please note the difference between the section “Connection to a production database” for Spring Boot 1.5.8 and the same section for Spring Boot 1.3.8!

For Spring Boot 1.3.8 we have:

And for Spring Boot 1.5.8 we have:

Now it got clear that parameters naming with “tomcat” was not valid for the Spring Boot version used in the project. The naming scheme was changed in Spring Boot somewhere between versions 1.3.x and 1.5.x! The correct form of this part of application.yml for Spring Boot 1.3.8 is:

spring:
  datasource:
    url: jdbc:mysql://localhost:3306/some_db
    username: root
    password: root
    driver-class-name: "com.mysql.jdbc.Driver"
    removeAbandoned: true
    removeAbandonedTimeout: 120
    logAbandoned: true
    maxActive: 50
    maxIdle: 5
    maxWait: 1000
    validationQuery: "select 1"
    testOnBorrow: true
    testOnConnect: true
    testWhileIdle: true

After this change the actual data source parameters dump provided by my diagnostic endpoint confirmed that now values from the configuration file are used.

Very tricky!

Conclusions

  1. One should always refer to the documentation in version corresponding to the version of a framework being used.
  2. It’s really hard to solve a problem without a diagnostic tool (like the InfoController class presented above) so don’t hesitate to invest time into preparing one. In a result it will save your time.
Posted in database, Java, Spring, Tomcat | Tagged , | 1 Comment

RestTemplate and <mvc:message-converters> in Spring Framework 4.3

The Spring Framework’s documentation says that RestTemplate instance created by the default constructor gets the default set of HttpMessageConverters.

But what is the default set of HttpMessageConverters? Well, this is defined in the Spring Framework Reference. Unfortunately I’ve missed it somehow and I was believing that the default set of HttpMessageConverters is affected by <mvc:message-converters> Spring XML configuration element, like in the sample below:

    <bean id="customGsonHttpMsgConverter" class="org.springframework.http.converter.json.GsonHttpMessageConverter">
        <property name="gson">
            <bean class="CustomGsonFactoryBean"/>
        </property>
    </bean>
    
    <mvc:annotation-driven>
        <mvc:message-converters>
            <ref bean="customGsonHttpMsgConverter"/>
        </mvc:message-converters>
    </mvc:annotation-driven>

    <bean id="restTemplate" class="org.springframework.web.client.RestTemplate"/>

Actually it’s not true. In the above example the RestTemplate bean will not be using this “customGsonHttpMsgConverter” bean. The default set of HttpMessageConverters for RestTemplate is… hardcoded in RestTemplate class. However one can customize it easily. This is the correct way of configuring HttpMessageConverters to be used by RestTemplate:

    <bean id="customGsonHttpMsgConverter" class="org.springframework.http.converter.json.GsonHttpMessageConverter">
        <property name="gson">
            <bean class="CustomGsonFactoryBean"/>
        </property>
    </bean>
    
    <bean id="restTemplate" class="org.springframework.web.client.RestTemplate">
        <constructor-arg>
            <list>
                <ref bean="customGsonHttpMsgConverter"/>
            </list>
        </constructor-arg>
    </bean>

Lesson learned! RestTemplate has nothing to do with <mvc:message-converters> configuration.

Posted in Java, Spring | Tagged | Leave a comment

How to add a key binding to toggle a touchpad under Linux

A laptop's key toggling the touchpad My previous laptop (Asus) had a special key on its keyboard to toggle the touchpad. It was working out of the box with Linux Mint & the Mate desktop environment – there was a special support for it. My new laptop (HP) doesn’t have this special key and I was really missing it. This article is about how to add a key binding under Linux that will be behaving exactly as having such a special key (tested on Ubuntu 16.04 LTS and Cinnamon 3.4.6).

Let’s identify what is a keycode that is recognized by a desktop environment as a signal to toggle the touchpad. To discover this run the command: xmodmap -pke|grep -i touchpad
On my computer results look as follows:

$ xmodmap -pke|grep -i touchpad
keycode 199 = XF86TouchpadToggle NoSymbol XF86TouchpadToggle
keycode 200 = XF86TouchpadOn NoSymbol XF86TouchpadOn
keycode 201 = XF86TouchpadOff NoSymbol XF86TouchpadOff

So now we know keycode 199 is the one recognized as toggling the touchpad.

Now we need a program that can simulate pressing a key with such a keycode. I found this can be done by xdotool. I needed to install it with the following command:

sudo apt install xdotool

Now you can test the program with the command:

xdotool key 199

For me it worked like a charm.

Finally we need to setup a key binding that will run the above command actually toggling the touchpad. In the Cinnamon desktop environment one does this as follows (optionally you can refer the section 15. Custom Keyboard Shortcuts of How To Change The Linux Mint Cinnamon Keyboard Shortcuts):

  1. Open system settings window
  2. Click on “Keyboard” item
  3. Go to “Shortcuts”
  4. Click on “Add custom shortcut” button
  5. Enter a name for a key binding (something like “touchpad toggling”)
  6. Enter a command: xdotool key 199
  7. Click on “Add” button
  8. Now select the first “unassigned” item in “Keyboard bindings” section and click it again to activate key capturing
  9. Press a key or a key combination of your choice

I used just F5 key for this and it’s working great. I have to admit that initially I wanted a key combination of windows-key+F5 but it was not working, but this is some issue with windows-key (it’s named SUPER key under Linux) and Cinnamon.
In my opinion this case is another example of how elastic Linux and open source software are. And it’s great!

Posted in Linux | Leave a comment