SPRING 3.0 MIGRATION OVERVIEW

Spring Boot 3.0 has just been released, packed with exciting new features such as advanced observability, better support for native compilation with Graal, and the ability to utilize the latest Java language features. However, for many individuals, upgrading to this latest version may appear overwhelming. The reason being, it requires the updating of many other aspects, such as the Java versions and a transition to Jakarta EE 9. Additionally, given the probability of your organization running several different dependency versions, the migration process can become even more complicated.

It is common to adopt an approach that revolves around the ideology of “if it’s not broken, why fix it?” However, it is important to note that the support for Spring Boot 2.x is scheduled to end by the end of 2023. Additionally, upgrading to the latest version of Spring Boot will provide you with access to a range of new features available across different tools. For instance, upgrading to Java 17, a prerequisite for Spring Boot 3.0, will not only grant you access to a variety of new Java language features (such as pattern matching, records, switch expressions, etc.), but it will also improve the performance of the virtual machine and garbage collector.

MAIN STEPS

Migrating to Spring Boot 3.0 necessitates a series of associated migrations and dependency updates that must be carried out before upgrading to this latest version of Spring Boot. These include:
• Updating your organization’s applications, infrastructure, and CI/CD pipeline to use Java 17. The upside to this step is that it can be done before updating any of your Spring Boot applications.
• Any pre-existing Spring applications that utilize Java EE will require an update to Jakarta EE 9. While this may seem like a simple task of moving all imports from the javax namespace to the jakarta namespace, it also requires the migration of any third-party libraries to versions that are compatible with Jakarta EE 9.

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.validation.Valid;
->
import jakarta.servlet.ServletException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.validation.Valid;

There are instances when we may not need to refactor package names when upgrading our software, as some dependencies such as javax.sql.DataSource already belong to JDK 17, not Jakarta EE. In these cases, it is not necessary to make any changes to the package name, as it is already consistent with the latest version of our app. It is important to note, however, that it is essential to thoroughly review and test all dependencies to ensure that they are compatible with the new version of the code.

• Lastly, depending on which version of Spring Boot your applications are being migrated from, there may be a need to make several required changes to both the application’s code and configuration when moving to Spring Boot 3.0.
It is crucial to take note of these steps, as they are all essential requirements that must be fulfilled before upgrading. Sometimes we need to just upgrade dependency version, sometimes we have to migrate to a new package. For example:

<dependency>
<groupId>org.springdoc</groupId>
<artifactId>springdoc-openapi-ui</artifactId>
<version>1.6.11</version>
</dependency>
->
<dependency>
<groupId>org.springdoc</groupId>
<artifactId>springdoc-openapi-starter-webmvc-ui</artifactId>
<version>2.0.0</version>
</dependency>

POST MIGRATION ISSUES – COMMON EXAMPLES

#1

Now let’s look at a situation where we want to update a library used by other applications as a Maven dependency.

Parameter 0 of constructor in com.some.package.a.Class required a bean of type 'com.some.package.b.SomeExternalClass' that could not be found.

There is a good chance that exporting configuration files is the cause. Before 3.0, Spring Boot checked for the presence of a META-INF/spring.factories file within your published jar. The file should list your configuration classes under the EnableAutoConfiguration key. Now, instead of providing configuration in META-INF/spring.factories, you would do so in META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports.

org.springframework.boot.autoconfigure.EnableAutoConfiguration=pl.inero.commons.email.EmailServiceConfiguration
->
pl.inero.commons.email.EmailServiceConfiguration

We’llonfiguration properties for Redis have moved from spring.redis. to spring.data.redis. as redis auto-configuration requires Spring Data to be present on the classpath. Also configuration Properties for Cassandra have moved from spring.data.cassandra. to spring.cassandra..

Now let’s take some Spring Security related issue.

#2

Invalid CSRF token found for http://localhost:8080/api/test
Responding with 403 status code

When using Spring Security, it’s important to be aware of changes between different versions of the framework. For example, in Spring Security version 5, the CsrfToken is loaded by default on each request, which means that even unnecessary requests must have the HttpSession read in a typical setup. However, the default behavior of Spring Security 6 is to postpone the lookup of the CsrfToken until it is actually required.

In our particular application, the CsrfToken is required for every request, so we need to make sure that we specifically opt into the 5.8 defaults to ensure that the CsrfToken is loaded on every request, as it was in version 5. This is a necessary step to ensure that our application runs as intended and that we do not encounter any issues related to the CsrfToken during runtime.

@Bean
public SecurityFilterChain securityFilterChain(HttpSecurity http) throws Exception {
return http
/ ... /
.csrf(csrf -> csrf
.csrfTokenRepository(CookieCsrfTokenRepository.withHttpOnlyFalse()))
/ ... /
}
->
@Bean
DefaultSecurityFilterChain springSecurity(HttpSecurity http) throws Exception {
CsrfTokenRequestAttributeHandler requestHandler = new CsrfTokenRequestAttributeHandler();
requestHandler.setCsrfRequestAttributeName(null);
return http
/ ... /
.csrf((csrf) -> csrf
.csrfTokenRepository(CookieCsrfTokenRepository.withHttpOnlyFalse())
.csrfTokenRequestHandler(requestHandler)
/ ... /
}

#3

Cannot resolve method 'antMatchers()' (...)

Methods like antMatchers() (as well as mvcMathcers() and regexMatchers()) have been deprecated and removed with Spring Security 6.0 (have a look at this link if you wonder why: Deprecate trailing slash match). Overloaded method requestMatchers() was provided as a uniform mean for securing requests. It facilitates all the functionality of the configuration methods that have been removed from the API.

Also, method authorizeRequests() has been deprecated and shouldn’t be used anymore. A recommended replacement – authorizeHttpRequests() (you can find more information regarding these changes here). Here we have sample code and how it could have been changed:

http.requestMatcher(new NegatedRequestMatcher(new AntPathRequestMatcher(API_KEY_URI_MATCHER)))
.httpBasic().and()
.authorizeRequests()
.antMatchers("/ping").permitAll()
.anyRequest().authenticated();

->

http.securityMatcher(new NegatedRequestMatcher(new AntPathRequestMatcher(API_KEY_URI_MATCHER)))
.httpBasic().and()
.authorizeHttpRequests()
.requestMatchers("/ping").permitAll()
.anyRequest().authenticated();

ON CONFIGURATION PROPERTIES

When upgrading to Spring Boot 3.0, it is essential to update the configuration properties in the application.properties or application.yml files to account for any changes or removals of properties. Spring Boot provides a convenient tool called spring-boot-properties-migrator to help with this migration. By adding this module as a dependency to our project, it will migrate the properties temporarily at runtime and analyze the application’s environment to print diagnostics at startup.

To add the spring-boot-properties-migrator module to our project, we need to add the following dependency to the pom.xml file.

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-properties-migrator</artifactId>
<version>3.0.0</version>
<scope>runtime</scope>
</dependency>

This will ensure that our properties are migrated correctly and that we can benefit from any diagnostics provided by the module during startup.

As always, more tips on migration can be found in the Spring documentation. Check it out now.

In Inero Software we build digital products for digital logistics of tomorrow. We connect active aproach to analysis of business needs and combine with cutting edge technologies and AI/ML. In the blog post section you will find other articles about IT systems and more!

Artykuł Spring Boot 3.0 migration overview pochodzi z serwisu Inero Software - Software Consulting.

Artykuł Building a custom voice command recognition system pochodzi z serwisu Inero Software - Software Consulting.

class MyDeviceAdminReceiver : DeviceAdminReceiver() {
    companion object {
        fun getComponentName(context: Context): ComponentName {
            return ComponentName(context.applicationContext, MyDeviceAdminReceiver::class.java)
        }
    }
}

<application>
          ...
      <receiver
            android:name=".MyDeviceAdminReceiver"
            android:description="@string/app_desc"
            android:label="@string/app_name"
            android:permission="android.permission.BIND_DEVICE_ADMIN">
            <meta-data
                android:name="android.app.device_admin" />
            <intent-filter>
                <action android:name="android.app.action.DEVICE_ADMIN_ENABLED" />
            </intent-filter>
        </receiver>
    </application>

override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        mDevicePolicyManager = getSystemService(Context.DEVICE_POLICY_SERVICE) as DevicePolicyManager
        val isAdmin = mDevicePolicyManager.isDeviceOwnerApp(packageName)
        if (isAdmin) {
            println("You are an admin")
        } else {
            println("You are not an admin")
        }
    }

private fun enableStayOnWhilePluggedIn(active: Boolean) = if (active) {
        mDevicePolicyManager.setGlobalSetting(mAdminComponentName,
                Settings.Global.STAY_ON_WHILE_PLUGGED_IN,
                (BatteryManager.BATTERY_PLUGGED_AC
                        or BatteryManager.BATTERY_PLUGGED_USB
                        or BatteryManager.BATTERY_PLUGGED_WIRELESS).toString())
    } else {
        mDevicePolicyManager.setGlobalSetting(mAdminComponentName, Settings.Global.STAY_ON_WHILE_PLUGGED_IN, "0")
    }

mDevicePolicyManager = getSystemService (Context.DEVICE_POLICY_SERVICE) as DevicePolicyManager

private fun setRestrictions(disallow: Boolean) {
        setUserRestriction(UserManager.DISALLOW_FACTORY_RESET, disallow)
        setUserRestriction(UserManager.DISALLOW_ADD_USER, disallow)
        setUserRestriction(UserManager.DISALLOW_MOUNT_PHYSICAL_MEDIA, disallow)
        setUserRestriction(UserManager.DISALLOW_ADJUST_VOLUME, disallow)
        mDevicePolicyManager.setStatusBarDisabled(mAdminComponentName, disallow)
    }
private fun setUserRestriction(restriction: String, disallow: Boolean) = if (disallow) {
        mDevicePolicyManager.addUserRestriction(mAdminComponentName, restriction)
    } else {
        mDevicePolicyManager.clearUserRestriction(mAdminComponentName, restriction)
    }

private fun setAsHomeApp(enable: Boolean) {
        if (enable) {
            val intentFilter = IntentFilter(Intent.ACTION_MAIN).apply {
                addCategory(Intent.CATEGORY_HOME)
                addCategory(Intent.CATEGORY_DEFAULT)
            }
            mDevicePolicyManager.addPersistentPreferredActivity(
                    mAdminComponentName, intentFilter, ComponentName(packageName, MainActivity::class.java.name))
        } else {
            mDevicePolicyManager.clearPackagePersistentPreferredActivities(
                    mAdminComponentName, packageName)
        }
    }

private fun setImmersiveMode(enable: Boolean) {
        if (enable) {
            val flags = (View.SYSTEM_UI_FLAG_LAYOUT_STABLE
                    or View.SYSTEM_UI_FLAG_LAYOUT_HIDE_NAVIGATION
                    or View.SYSTEM_UI_FLAG_LAYOUT_FULLSCREEN
                    or View.SYSTEM_UI_FLAG_HIDE_NAVIGATION
                    or View.SYSTEM_UI_FLAG_FULLSCREEN
                    or View.SYSTEM_UI_FLAG_IMMERSIVE_STICKY
                    )
            window.decorView.systemUiVisibility = flags
            window.decorView.setOnSystemUiVisibilityChangeListener {
                window.decorView.systemUiVisibility = flags;
            }
        } else {
            val flags = (View.SYSTEM_UI_FLAG_LAYOUT_STABLE
                    or View.SYSTEM_UI_FLAG_LAYOUT_HIDE_NAVIGATION
                    or View.SYSTEM_UI_FLAG_LAYOUT_FULLSCREEN)
            window.decorView.systemUiVisibility = flags
        }
    }

private fun setLockTask(start: Boolean, isAdmin: Boolean) {
        if (isAdmin) {
            mDevicePolicyManager.setLockTaskPackages(
                    mAdminComponentName, if (start) arrayOf(packageName) else arrayOf())
        }
        if (start) {
            startLockTask()
        } else {
            stopLockTask()
        }
    }

In Inero Software – software development agency, we are experts of digital transformation and smart automation. We provide knowledge and expertise on how to successfully use cutting edge technologies and data to shape corporate digital products of the future. Find out more visiting our website or contacting us directly on hi@inero-software.com

Artykuł Android Kiosk Mode – how to turn an Android device into a single-use device pochodzi z serwisu Inero Software - Software Consulting.

Computer vision and object detection are increasingly used in the automation of business processes. Along with the dynamic development of technology, especially artificial intelligence, there are many new innovative business applications for this type of algorithms. In one of our recent posts, we outlined how to build a custom R-CNN based detector. In this article, we will walk you through how to do the same with YOLO v5, which has grown in strength in recent years. One of the important aspects over and above other solutions is the speed of inference. As part of this article, we will present the specified use case and all steps of its implementation. Enjoy your reading!

Use case

Signatures are still one of the most common methods of document authentication. Especially for enterprise applications, identification of signed and unsigned copies of documents in digital repositories may be time-consuming and a challenging task. However, automation supported by machine learning can make it easier. In this context, we describe the capabilities of YOLO v5 detector, and we will discuss issues of detecting invoice signatures. Let’s start with an example as shown in the figure below. For the purpose of this study, we prepared a fake invoice, fake data of the seller, buyer or the product itself. Let’s investigate now, how we can focus on hand-written signatures (which is fake too :-)).

Collecting Data

One of the first steps is to collect a set of images to train your model. For our use case, we have prepared a small 10 sample training set of invoices filled with different data and augmented this collection by a dedicated Python script. To make things easier, we used a Roboflow, which is a useful tool for data tagging Here you can upload your dataset and make a quick annotation process like on this GIF.

Additionally, we can generate more output images with random values ​​of rotation, saturation, exposure, noise, blur and other types of transformations. Moreover, with the help of this tool, we can determine our training, validation and test data split – the default is 70%, 10%, 10%. This is an important thing to prevent overfitting our model, you can read more about it on this blog. Thanks to the described tool, we can export our dataset in the YOLO v5 Pytorch format and put it in the directory of our project. The figure below shows the selection and target tree, where one of the most important files is the data.yml that will be used for training.

In our case after the first step of augmentation by a Python script, we generate 69 images. Additionally, we use a noise option in Roboflow, which allows us, in the end, export 169 annotated examples. Our images were ultimately split as shown in the figure below. 

And here you can see some of the train examples… 

Model configuration and architecture

The next step in the whole process is to define the configuration and architecture of the YOLO model. We may build our own network structure, although in version 5 we are provided with one of the following models:

• YOLOv5s,
• YOLOv5m,
• YOLOv5l,
• YOLOv5x.

We may use any of them but we must remember to assign appropriate target file parameter value called nc – number of classes. In our use case, it’s 1. The models differ from each other with the number of parameters used, speed of frames per second (FPS), accuracy and others… On the figure below you can see how they cope with the same COCO dataset. Click on the image below to learn more.

Training

What’s important, the Ultralytics company provides us with YOLO v5 developed in PyTorch, a framework specialized in machine learning. Thanks to this, we may download their repository from GitHub and train our own detector. Nothing simpler, but what else do we need for everything to work fine? 

To run each of the scripts we must install the dependencies contained in the requirements.txt. If we are using pip, we can use the following command in the terminal. Please remember that you must be in the project directory. 

pip install -r requirements.txt

In the installation process, one thing may be a problem. Namely, the PyTorch library…

The different operating system, package in use, programming language etc. may require a specified command. You will find everything you need in this tutorial.

Let’s assume you have passed all the requirements and start our training!

One line of code is enough to run this process, but we need to make sure that we have prepared two important files – data.yaml and for example yolov5l.yaml. The first one you should have from the data collection stage and the second one can be found in the YOLO v5 repository, in the Models directory. 

That’s it! Let’s run the training by the following command: 

python train.py --data dataset/data.yaml --cfg models/yolov5l.yaml --weights ''

The above command is the simplest possible, additionally, you can define the following options or parameters: 

• img-size,
• batch-size,
• epochs, 
• name,
• no-save,
• cache…

For the purposes of this article, we conducted a relatively short training for images resized to 416×416, batch size 32 and 1500 epochs. When the process is complete, the result will be a weight file named best.pt.

Perhaps you may stop your learning process if your weights are optimal for you at the moment. We stopped our training after 650 epochs, which took about 15 hours in total. Please note that the PC used does not have a dedicated graphics card.

Detailed metrics of our training:

• precision – 0.93507,
• recall – 0.96429,
• mAP_0.5 – 0.94755,
• mAP_0.5:0.95 – 0.48702,
  

where:

precision – measures how accurate are your predictions,

recall – measures how good you find all the positives,

mAP_0.5 – mean average precision for IoU* = 0.5,

mAP_0.5:0.95 – mean average precision for IoU* from 0.5 to 0.95 with a step size of 0.005,

* IoU (Intersection over Union) – measures the overlap between 2 boundaries. Find out more about these metrics here.

Inference on test images

Now, when we have our trained model trained after the learning stage, we may go to inference on test images. As you remember, one of the directories exported from Roboflow was called test. We can use it by putting in the terminal following line: 

python detect.py --weights best.pt --source dataset/test --conf 0.6 --img-size 600

where conf is model confidence – higher required makes fewer predictions. 

Finally, we may see the result visualization. The approximate inference time on one test image 416×416 was ~ 0.3 s, while for 700×700 ~ 0.8s.

Summary

For the implementation of the described detector, we used only 10 images, extending the dataset by augmentation process. Thanks to the Roboflow tool, it was possible to quickly annotate and export the data to the YOLO format. The learning process allowed us to obtain a target detector that effectively recognizes signatures on invoices.

Please note that with such a small dataset it is only adapted to recognize similar invoices. If we wanted to expand the possibilities of our detector, we would have to equip ourselves with a better graphics card and more, and more data. Additionally, you may be tempted to add a new class, e.g. to separate the objects into the Signature of the Seller and the Buyer.

Maybe you will see a new article in the future with more recognition classes. Stay tuned!

In Inero Software – software development agency, we are experts of digital transformation and smart automation. We provide knowledge and expertise on how to successfully use cutting edge technologies and data to shape corporate digital products of the future.

For more information, visit us on our website or follow us on LinkedIn. 

Artykuł Digital Cloud Document Repositories – how to identify signatures in scanned PDF documents pochodzi z serwisu Inero Software - Software Consulting.

public virtual async Task BuildUserRolesViewModel(TKey id, int? page)
{s
    var roles = await GetRolesAsync();
    var userRoles = await GetUserRolesAsync(id.ToString(), page ?? 1);
    userRoles.UserId = id;
    userRoles.RolesList = roles.Select(x => new SelectItemDto(x.Id.ToString(), x.Name)).ToList();

    return userRoles;
}

public virtual async Task GetUserRolesAsync(string userId, int page = 1, int pageSize = 10)
{
    var userExists = await IdentityRepository.ExistsUserAsync(userId);
    if (!userExists) throw new UserFriendlyErrorPageException(string.Format(IdentityServiceResources.UserDoesNotExist().Description, userId), 
    IdentityServiceResources.UserDoesNotExist().Description);

    var userIdentityRoles = await IdentityRepository.GetUserRolesAsync(userId, page, pageSize);
    var roleDtos = Mapper.Map(userIdentityRoles);

    var user = await IdentityRepository.GetUserAsync(userId);
    roleDtos.UserName = user.UserName;

    await AuditEventLogger.LogEventAsync(new UserRolesRequestedEvent(roleDtos));

    return roleDtos;
}

Artykuł Skoruba for Identity Server – a review of authorization module for .NET Applications pochodzi z serwisu Inero Software - Software Consulting.

WARNING: I assume that you know basics of asynchronous programming, if not then please read this before going any further: Asynchronous Concepts

So, what’s RxJs?

According to the documentation:

RxJS is a library for composing asynchronous and event-based programs by using observable sequences. It provides one core type, the Observable, satellite types (Observer, Schedulers, Subjects) and operators inspired by Array#extras (map, filter, reduce, every, etc) to allow handling asynchronous events as collections.

In simple words, RxJs is a library which helps you in working with asynchronous tasks, events and streams and it does that by sharing some useful classes like aforementioned Observable and a ton of cool operators (due to my desire of keeping this post relatively short I am going to present only 5 operators that I find the most useful in my everyday work). Actually, RxJs has been so helpful to the Angular community that the Angular team decided to include it into Angular itself with the 6th version.

We are going to focus on Observables this time. Observable is a class that allows you to listen for events. To retrieve data from an observable you need to subscribe to it, when you do that it listens for events happening, so when they happen, you get the data. You can get more details on observables here: https://rxjs.dev/guide/observable.

Now let’s discuss 3 most common situations where developers are using Observables

Http Requests

All HtttpClient methods are returning Observables and that is of course because it takes time for a server to process and return the data (or just return in case you’re sending a get request). That means that http request operation is asynchronous.

DOM events

Observables enable you to easily listen for DOM events like click or keyup. To listen to them you should use the fromEvent function that creates an observable accordingly. Here’s how this looks like:

fromEvent(document, ‘keyup’);

(Remember that if you want to receive the actual data you should subscribe to it by adding .subscribe() before semicolon)

Stores

Another very common use case for using Observables is using them with some kind of store, like NgRx or NgXs. Stores like that usually share observables in order to keep watch over the state of the application.

Now since you know the very basics like why and how, let’s get to the useful interesting stuff that can be one with RxJs operators.

RxJs operators

In our examples we are going to use simplest observable possible:

const obs$ = from([1, 2, 3]); (dollar sign at the end of the variable is a naming convention used to singlaize that the variable is indeed an observable)

How to use operators?

Easiest way to use operators is by putting them inside a pipe. Pipe runs a function for every stream element passing by the operator, and operators are the functions that can be used within it.

first

First operator is the first operator that we’re going to talk about (got the pun?). Most operators have self-describing titles and this case is no different. First operator basically allows you to listen for the first returned value of the observable and then stop (note that it is completely useless combined with Http requests that return the value once anyways)

obs$.pipe(first()).subscribe(numbr => console.log(number));

Code above prints only 1 because it’s the first event emitted, 2 and 3 are ignored.

tap

Another useful operator worth mentioning is tap. Tap allows you to perform some kind of side effect for every event emission. For example you can log something or assign data to a variable (note: if you are using .subscribe it is basically the same thing).

obs$.pipe(tap(numbr => console.log(number))).subscribe();

Code above prints 1, 2 and 3 because tap is triggered for every emitted value.

map

Map is working the same way as in arrays. So it basically allows you to change structure of returned data. For example, let’s try to multiply every value returned by 2

obs$.pipe(map(numbr => numbr * 2)).subscribe(value => console.log(value));

The code prints 2, 4 and 6 just as expected.

startWith

StartWIth allows you to decide what first value is going to be. It is useful when you want to trigger the observable before the first event emission.

obs$.pipe(startWith(999)).subscribe(value => console.log(value));

Printed values are: 999, 1, 2, 3.

switchMap

SwitchMap is used mainly in typeahead cases but not limited to them. What it does – it cancels previous request when a new event arrives. In real life case you would for example listen for changes on the input and filter a table while typing (previous result is not relevant).
To present switchMap fully – this time we are going to use a different example. Observable is going to be created out of DOM event keyup.

const another$ = timer(1000);
fromEvent(document, 'keyup').pipe(
switchMap(() => another$)
).subscribe(() => console.log('log'));

What happens here is: After you click a button on keyboard, wait 1000 ms and then log a message. Now what happens ,if you click button 10 times quicker? Only one message is going to be logged, and that is because switchMap cancels previous timers leaving only the most recent one.

We went over 5 most useful operators in my opinion, described what RxJs is and discussed common use cases. I hope that this article will help you in your next project and interest you in exploring RxJs more 🙂

Inero Software provides knowledge and expertise on how to successfully use cutting edge technologies and data to shape corporate digital products of the future.

In the blog post section, you can find other articles about our projects and more! 

Artykuł RxJs for beginners (5 most useful operators) pochodzi z serwisu Inero Software - Software Consulting.

SchedulerFactory schedulerFactory = new StdSchedulerFactory();
Scheduler scheduler = schedulerFactory.getScheduler();

public class FileDownloadJob extends QuartzJobBean {

   @Override
   protected void executeInternal(JobExecutionContext jobExecutionContext) throws JobExecutionExceptions{
       // target code of performed operations
   }
}

JobKey jobKey = JobKey.jobKey((task.getId());

JobDataMap jobData = new JobDataMap();
jobData.putAll(new HashMap<>() {{
 	  put("email", user.getEmail());
  	  put("name", user.getName());
          put("progress", task.getProgress());
}});

JobDetail jobDetail = newJob().ofType(FileDownloadJob.class)
       				.withIdentity(jobKey)
      				.setJobData(jobData)
      				.build();

Trigger trigger = newTrigger()
       .startNow()
       .build();

scheduler.scheduleJob(jobDetail, trigger);

public class FileDownloadJob extends QuartzJobBean implements InterruptableJob

@Override
public void interrupt() {
   _interrupted = true;
}

if (_interrupted) {
   break;
}

scheduler.interrupt(jobKey);

jobDataMap.put("nextJob", nextTask.getId());

Artykuł Java: Scheduling the execution of background tasks using Spring pochodzi z serwisu Inero Software - Software Consulting.

Thanks to the development of computing technology and artificial intelligence in recent years, applications using machine learning and systems based on neural networks have more and more business uses. Moreover, the advancement of technology has made machine learning, adapting advanced solutions based on neural networks cheaper and allows for relatively quick creation of solutions that have direct application, such as tracking objects on the camera image (tracking cars in parking lots, detecting people on construction sites) or extracting information from scans of documents (finding amounts and contractor data on an invoice). 

In the described material, we presented the methodology that we use as part of our ML-based work using R-CNN networks, which due to the universality of the structure and architecture of the model, work well in a wide range of areas. It is a type of neural network specifically designed to detect objects, which at least one layer is a convolution layer, allowing to distinguish the features of the manipulated image. Based on the input image, R-CNN determines potential ROI’s (regions of interest) by selective search segmentation. Then each of them is analyzed by a neural network to obtain the probability of the object classification. 

R-CNN detectors are widely used in: 

• driver assistance systems in autonomous cars, 
• face recognition, 
• anomalies detection,  
• health risk assessment and biomarkers of aging discovery, 
• Intelligent multi-site surveillance system, 

and many others… 

Building your own R-CNN detector is a simple task that does not require a lot of work. In this article you will see an example of its use and the next steps in the process. 

Use case

In order to demonstrate the possibility of building Your own detector based on R-CNN, the use case in this article will be the date detection on the transport document. Your solution may have different elements or the number of classes of objects to be recognized. A sample document is shown in the illustration below. As You can see, there are many irrelevant elements on this picture. In this case, we are only interested on each date. 

Software architecture

Programming tools enabling the implementation of the system in Python are, for example: 

• Tensorflow, 
• Keras, 
• OpenCV, 
• Scikit-learn. 

Once we have configured development environment, the first step in building the system is collecting the dataset. In order to make a R-CNN based detector with a high measure of accuracy, remember that: 

• the collection of images must be extensive and diverse, 
• as input to the model must be of the same dimension,
• eliminate duplicates that can cause network over-matching. 

For a good start, in the described case of the article it is enough to prepare even a few photos of the elements that we want to recognize, preferably against the background of other objects. Why such preferences? In the next step of the algorithm, should be a positive and negative samples generated from the prepared set. 

In order to generate the samples, we will use selective search segmentation.  The possibility of using this algorithm is provided by the OpenCV library.  This function uses hierarchical clustering to group pixels and then combine them into one based on color, texture or composition. The following is an implementation of the search code on the base image. The output of the process is a set of a potential ROI’s, depending on the size of the photo, even over 2.000 item’s.  

selective_search = cv2.ximgproc.segmentation.createSelectiveSearchSegmentation()
selective_search.setBaseImage(image)
selective_search.switchToSelectiveSearchFast()
rects = selective_search.process()

To divide the generated ROI’s, for example, we can use a metric called IoU (Intersection over Union). It’s defined as the intersection area divided by area of the union of a predicted bounding box and ground-truth box. Using this factor will allow You to filter when building a sample set. However, before we proceed to this operation, You should manually prepare the ground truth.  There are many different object tagging programs available on the internet. In this article, we will use the VoTT (Visual object Tagging Tool) originally developed by the Commercial Software Engineering (CSE) group at Microsoft.  

The above image shows the tagged date in the sample photo.  The operation should be repeated for all prepared data and exported to the appropriate format (JSON, XML, etc.). The generated files contain the exact coordinates and labels of previously marked objects. This allows us to compare our tagged data with those that will return a selective search segmentation. In this algorithm, we decided to set the IOU to 0.5, because of the good prediction result. You can modify this ratio depending on Your preferences and check whether the resulting data set is consistent with the assumptions. After executing the script, the filtered data should look like this (on the left – positives, on the right – negatives).  

As You can see, the positive samples present a variety of dates, while the negative other image elements.  On such a dataset, we can move on to building the R-CNN neural network model. A helpful library in this aspect will be the previously mentioned Tensoflow with Keras.  

After loading samples of the created model architecture You should split into test and training data. We can quickly do this with a single line of code using scikit-learn, when we can define the size of the test data (test_size) and the shuffle value (random_state). 

(self.train_x, self.test_x, self.train_y, self.test_y) = train_test_split(self.data, self.labels,
                                                                                  test_size=0.1,
                                                                                  stratify=self.labels,
                                                                                  random_state=42)

The Tensorflow library is already so extensive that it provides us with ready-made network architectures. The implemented algorithm uses the MobileNetV2 model, containing pre-trained weights from ImageNet. Here we define the dimension of each tensor, i.e. in this case the image, which is the input of the model. 

self.model = MobileNetV2(weights="imagenet", include_top=False, input_tensor=Input(shape=(224, 224, 3)))

To the ready model architecture, optionally we can add layers to improve the network. In this use case have been used: 

• pooling – combining the spatial data created by the convolution layer, minimizing the input data in both dimensions, 
• flatten – minimazing each tensor to one dimensional, 
• dense – enabling the connection of nodes in each subsequent layer using the activation function, 

• droput – consisting in a random data rejection. Allows for regularization to prevent overfitting. It also proved the flexibility of the neural network and the margin of response to input data.  

The next step in creating Your own R-CNN based detector is to compile the model with defined optimizer, metrics and loss function.  

self.model.compile(loss="binary_crossentropy", optimizer=Adam(lr=self.lr), metrics=["accuracy"])

This algorithm uses the Adam’s optimizer, the acurracy metric and the loss function binary crossentropy. The selected parameters are popular for similar use cases, but in the detailed Tensorflow documentation you can find something specific for Your solution.  

Now we can proceed to the training process. By properly defining the parameters of a function, we can do it with a line of code:  

self.model.fit(
    x=None, y=None, batch_size=None, epochs=1, verbose=1, callbacks=None,
    validation_split=0.0, validation_data=None, shuffle=True, class_weight=None,
    sample_weight=None, initial_epoch=0, steps_per_epoch=None,
    validation_steps=None, validation_batch_size=None, validation_freq=1,
    max_queue_size=10, workers=1, use_multiprocessing=False
)

Remember to choose the right learning parameters. Each epoch is modified with indivudal weights, which affects the quality of the network. In the algorithm results section of this article, You can see the parameters we selected and information on how to modify them. 

After completing the network training process, it is enough to save the model to a file and proceed to the next stage, i.e. object detection on test data. Now, for each input image, we once again perform a selective search segmentation and use the built-in model to predict individual ROI’s. For our own solution, we can define a minimum probability for which the regions will be considered valid. Typically, this value is in the range of 95-99%. Finally, all ROI’s that were filtered out in the previous operations can be marked on the target image by using simple function of the OpenCV library. The next chapter presents the effects of the R-CNN based detector constructed in this article. 

Results and summary

To implement detecor in this article were used only 10 images. It allowed to generate about 150 positive and 1000 negative samples. An important aspect of the network training itself is the selection of parameters. These include learning rate, batch size and number of epochs. The Tensorflow library provides tools for evaluating the solution, such as Tensorboard or the classification report. In the first one, we can observe different types of graphs, such as the accuracy and loss of the learning process every epoch. Additionally, the report automatically predicts the test data, returning the final precision. Thanks to this, even if we select parameters experimentally, we can deduce their further modification in order to improve the network.  

The created dataset and selected parameters (learning rate = 0.00001, batch size = 32, epochs = 100) allowed to obtain the final accuracy – 97%. Considering that the image’s collection was relatively small, good results were achieved. At the end of this article, let’s show the effects of the implemented date detector on sample images of transport documents.  

You can see in the pictures that the detecor is doing very effectively. If we would like to expand the developed detector, we would have to: 

• prepare an extensive set of data, various date cases and their location on the document, 
• train the neural network again, 
• depending on the metrics used, modify the parameters of the learning process, 
• perform a detection on test images. 

By following a few steps outlined in this article, You can create Your own solution by classifying a variety of objects. You can even upgrade your detector by adding more recognition classes. 

Inero Software provides knowledge and expertise on how to successfully use cutting edge technologies and data to shape corporate digital products of the future.

In the blog post section you will find other articles about IT systems and more!

Artykuł Few tips on how to create custom class detection system using R-CNN pochodzi z serwisu Inero Software - Software Consulting.