Nut & Bolts of Unit Testing | VeteranCraft General Discussion | Forum

A A A

Please consider registering
guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —




— Match —





— Forum Options —





Minimum search word length is 4 characters - maximum search word length is 84 characters

sp_Feed Topic RSS dirt
Nut & Bolts of Unit Testing
Topic Rating: +1 (1 votes) 
December 11, 2016
7:27 pm
frelling
Senior Tech
Forum Posts: 3232
Member Since:
August 18, 2011
sp_UserOfflineSmall Offline

I figured that it would be best to put this in a separate thread rather than cluttering the Update thread with too much technical mumbo-jumbo. A couple members that have unit testing experience PMed me to ask for more details on how VC does unit testing and what had broken.

First and foremost, we develop all code – plugins, tools, database schema, etc. with Eclipse, Eclipse Mars to be specific. We had intentions to moved everything over to Eclipse Neon earlier this year, but given the lateness, we’ve stuck with Mars to avoid any potential problems introduced by a newer IDE.

Unit tests are run on a plugin-by-plugin basis within Eclipse to test critical functions and features that can easily be tested procedurally. We use Corbetura to create code coverage reports allowing us to identify areas that might be candidates for more unit tests. Although we’d like to strive for 100% coverage, there is only so much we can reasonable test without necessitating an overhaul of Bukkit/Spigot to be more unit testing friendly. We just treat it as legacy code.

All code is maintained in a central, private Git repository known as SCM Manager. This is tied in with a master Jenkins server and several slaves (we test more than just Minecraft stuff Kiss) upon which we run more elaborate unit tests such as intra-plugin tests and integration tests with our backend services (i.e. database, management system, control scripts, etc.). Everything from there is fed into SonarQube for more code coverage, technical risk assessment, coding practices, and other coding metrics. Jenkins also spits out the customary JavaDocs, although we are slowly transitioning over to Doxygen (more features, more language coverage, etc.)

Unlike most plugin unit test examples found elsewhere, we do not fire up a full-fledged server for each test. If we took that approach, each of our tests would take several minutes to run rather than a few seconds. Multiple that by an average of 50 tests per plugin; it would quickly detract us from writing test code.

Instead, we use our own test library consisting of Mockito and PowerMock mock ups to create a virtual representation of a Bukkit server that is capable of handling players, worlds, events, permissions, and very basic plugin management. While this limits us to what can be tested, it is enough to ensure that plugins adhere to the JavaPlugin interface, Bukkit’s event-driven system and permissions system, etc.

The problem we ran into was due to our test environment not using SimplePluginManager for loading plugins. It is too cumbersome to mock the whole SimplePluginManager, JavaPluginLoader, and PluginClassLoader trio. Our focus is on testing a plugin and its interfaces, not how Bukkit manages and maintains plugins.

Instead we have our own loading routine that forgoes dependency checks, plugin prioritization, etc. It loads a plugin and related information (e.g. plugin.yml) directly from the test environment rather than a JAR file. However, that required us to use JavaPlugin’s protected constructor, rather than its empty constructor and a call to init().

Bukkit’s original plugin management system was designed to support a variety of plugin loading mechanisms, with a default for loading plugins from JAR files in the plugins directory. However, when we considered writing our own customer loader to load directly from the test environment, it became evident, thanks primarily to good ol’ Wesley Wolfe (Mr. DCMA himself), that the management routines were increasingly hard-coded to use JavaPluginLoader and PluginClassLoader. Never mind the co-dependencies that they introduced.

Instead, we were thankful for JavaPlugin’s protected constructor that allowed us to instantiate plugins directly, which took a PluginLoader (interface), Server, and other classes as arguments. The 1.11 Spigot update removed this constructor, and left us with a similar one that took a JavaPluginLoader and other classes as arguments.

That wasn’t so much of a problem, since we could just extend the class to do what we needed it to do. Sadly, JavaPluginLoader was marked as final; in other words, no extensions. Granted, there are always workarounds, but we wanted to stay as close to normal Java without using reflection techniques or bytecode manipulate to overcome that limit.

The solution to our dilemma was quite simple, though not initially obvious. We looked at JavaPlugin and how it used JavaPluginLoader. It turns out that it just saved a reference and provided a getter method to it; the only consumer of which was SimplePluginManager, which we didn’t use at all. Given that insight, we instantiated JavaPluginLoader and passed it to JavaPlugin’s constructor, knowing that it served no purpose other than to be a constructor argument. Once we updated constructor argument definitions for each of our plugins, everything started to work as expected again.

So there you have it. Another excursion into the bowels of Bukkit/CraftBukkit/Spigot. Now we can get back to focusing on normal plugin interfaces and events.

Human beings, who are almost unique in having the ability to learn from the experiences of others, are also remarkable for their disinclination to do so. - D. Adams
December 12, 2016
11:10 am
AdmlAdama
Member
Members
Forum Posts: 109
Member Since:
April 21, 2015
sp_UserOfflineSmall Offline

You're speaking my language now!  And yes... the siren call of 100% code coverage has crashed me a few times so good call there. Murphy's Law says you're next bug will be in that one piece of code you didn't check. 

Out of curiosity is there anything for monitoring individual modules memory use?  Of course most likely any memory leaks will happen when these things interact with each other and references are kept. 

Forum Timezone: America/New_York

Most Users Ever Online: 117

Currently Online:
5 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

Emulated: 3206

ryanpitts: 1300

Dalferes: 746

Pherian: 660

Okarim: 590

Member Stats:

Guest Posters: 10

Members: 1943

Moderators: 3

Admins: 2

Forum Stats:

Groups: 8

Forums: 45

Topics: 6215

Posts: 27283

Newest Members: Greogysealk, EROdonopay, DrongDib, BeverlyRot, SuzanneSak, Aisfelub, AbakbakoGex, ElyaLelia, Rebecajew, Mshertyuptex, Annated, JamesFlack, Rosejy, panlillita, Michaelelone, TheRealRedCakes, RedCakes, texpzofuncmasondpo, Rozeko, Jadoiretex

Moderators: terrorisly: 424, mudwog: 127, LightWarriorK: 2117

Administrators: meatbawllz: 2475, frelling: 3232