Showing posts with label maven. Show all posts
Showing posts with label maven. Show all posts

Tuesday, 19 February 2013

Android Project Templates

Today I'd like to share some templates I've made for creating Android apps. They all use Maven and have been coerced into compatibility with Eclipse. There are a few different flavours which I will describe below. If you want to skip the reading and just want the code, np, it's all here: https://github.com/japgolly/reference.

Template #1: Java

Code available here.

This is the gist of what I use when I'm using Android + Java. It comes with a few add-ons to mitigate Java's verbosity and inflexibility and the fact that they all work together in harmony, even with Eclipse, well that's what makes this template gold. It took a while to get working way back when.

Features
  • Lombok: Integrated into both Maven and Eclipse.
    Modified to play nice with Android, and added the ability to add @Inject to generated constructors.
    What is Lombok?
  • AndroidAnnotations: Integrated into both Maven and Eclipse.
    What is AndroidAnnotations?
  • CoFoJa: Contracts For Java provides annotations that can be used to verify pre-conditions, post-conditions and invariants. The validations are inherited so they're great for interfaces. Also they can be omitted from bytecode during compilation meaning you get more checking during tests and dev with no performance penalty in prod.
    More about CoFoJa.
  • Unit testing: Robolectric is being used for unit testing (along with an example of usage). I'm actually using a test library of mine (open-source) that includes Robolectric, FEST, Mockito and some Android-specific testing utilities.
    What is Robolectric?
  • Integration testing: Robotium is being used for integration testing (along with an example of usage).
    What is Robotium?
  • Android Lint: There's a profile that run Android Lint. Great for CI builds.
  • Findbugs: Findbugs has been integrated (not that it was effort). What's nice is that I've included a rules file to exclude certain Android-specific warnings. I'm a nice guy.
  • Proguard: Dev & release configs for Proguard are included.
  • Release: A release profile is included that runs Proguard, signs artifacts, and runs zipAlign.
  • AdMob: It's in there, ready to go but can also be deleted just as easily if you don't want it.
  • Eclipse settings: Auto-format on save. Auto-organise-imports on save. A TODOC tag for doco todos. More.

Template #2: Scala

Code available here.

Here you can use Scala instead of Java to build your app. The introduction of Scala renders obsolete the need for Lombok and AndroidAnnotations which simplifies the build process. As above, everything in this build works from CLI with Maven, and within Eclipse.

Features
  • Scala: Scala app code. Scala unit tests. Scala integration tests. Scala proguard. Scala Eclipse. Scala everything!
  • Unit testing: Robolectric is being used for unit testing (along with an example of usage). I'm actually using a test library of mine (open-source) that includes Robolectric, FEST, Mockito and some Android-specific testing utilities.
    What is Robolectric?
  • Integration testing: Robotium is being used for integration testing (along with an example of usage).
    What is Robotium?
  • Android Lint: There's a profile that run Android Lint. Great for CI builds.
  • Proguard: Proguard is mandatory when Scala is used. Rules included so that Scala works. Dev & release configs for Proguard are included.
  • Release: A release profile is included that runs Proguard, signs artifacts, and runs zipAlign.
  • AdMob: It's in there, ready to go but can also be deleted just as easily if you don't want it.

NOTE: You cannot use Scala and Java together because it wrecks Eclipse. If you're using VIM or some other editor inferior to mighty VIM, you'll be fine with both languages simultaneously, Maven can handle it, just don't expect a happy life using Eclipse.

Template #3: Scala + Free/Paid Editions of Your App

Code available here.

Same as the Scala template above except this time instead of having a single app, you have an Android library and two separate apps that extend it.

Features
  • Everything in template #2. See above.
  • Shared APK Library: Anything shared between your free & paid apps will live in the shared library. It's an Android library so resources can be shared too.
  • Separate Apps: Your free and paid apps are separate apps that extend the shared library. Additional code and/or resources can be dropped in as necessary.

Usage

To use one of these templates just copy the entire folder (and ensure you capture files starting with .). Next use the find_template_values.sh script to find places in the project where you need to swap out example values for real values. There's also a helper script, replace.sh that you can use to perform recursive, mass regex replacements.

To load your project into Eclipse, simply hit File → Import... → Existing Projects into Workspace. All the required Eclipse settings and files are already there in the template.

Enjoy.

Sunday, 10 February 2013

Releasing an App for Android - Part 2

Continuing on from part 1...

4. Free and Paid Editions.

The Google Play marketplace doesn't allow free & paid editions of apps. If you want free & paid editions you will need to manage and release two separate apps with separate package names.

To do this I decided to go with the mainstream approach, namely, creating a separate projects. If you have shared code without shared Android resources (unlikely) then you'd simply move the shared code out into its own project and include the resulting jar as a dependency in the free & paid projects. With Android it's similar, you create an APK library. Practically this means adding android.library=true to your library's project properties and changing its pom packaging from apk to apklib. The free/paid projects wont contain much but they will need their own AndroidManifests and a reference to the library in the project properties such as android.library.reference.1=../library. That's the gist of it.

One annoying thing is that, like a few things that have come out of the Android dev camp, it's a hack. The resulting library is basically just a zip of the resources and the SOURCE CODE. When you use an Android library in a top-level project it unzips, merges and recompiles the library code. How annoying. The reason is the generated R.java will produce different ids outside of the library, hence the recompilation. As far as I understand the system is therefore a quick hack. It might be fair enough - I don't know how much pressure the Android dudes are under or how well/poorly staffed they are. But technologically speaking is still a hack. Oh well. Make sure Scala incremental compilation is on as it takes a large bite of the build pain.

In retrospect it was a good thing that I saved this step until the end. The time to alter the build & structure (especially if you've got a working reference) near the end of the project will be much less than both the time spent by managing 3 separate projects instead of 1; and the time wasted by waiting on the extra compilation, dexing and proguard'ing.

Tip #1: Use a single project for as long as you can get away with but anticipate that towards the end you'll need a little time to split out library, free, and paid projects.

Tip #2: Anything edition-specific should be separated from anything shared. For example, create an ad banner layout and then <include> it rather than using an AdView directly. Later you can configure things so that the paid edition gets a NOP layout.

5. Advertising.

I started not knowing anything about ads. Basically I did this:

  1. Sign up with AdMob.
  2. Fill in all your bank details so they can pay you. You'll also need a merchant account with Google Play.
  3. Add your app but don't give it a market URL yet. This allows you to get an ID that you can start using before you release.
  4. AdMob doesn't just show you their own ads; they connect to other ad networks and can show you their ads too. That was surprising. Advantage us because it means that once you get ads working in your app and you want to use a different ad provider like InMobi or MobFox, you can just sign up with those guys, get your ID and plug it into AdMob on their website. No code changes required. You can even configure Admob to show ads from a combination of providers, or use different providers for certain countries.
  5. Get the AdMob SDK and wire it into your project.
  6. Follow the instructions on this page to integrate ads into your layout: https://developers.google.com/mobile-ads-sdk/docs/admob/fundamentals. Reading all those pages is a good idea.
  7. Finally, if you want ads to refresh every 60 sec or so, don't do it in code; play around in the AdMob website and you'll find settings for it. You can also customise the appearance of ads on the website or in code.
Done.

Now I don't know much about how it all works yet but this is what I've gleamed:

  • Every time an ad is shown to a user it's called an “impression”. You get nothing, $0.00, for impressions.
  • Every time an ad is clicked you get something small like $0.03.
  • RPM = “Revenue Per Mile” = the amount of money you've made per 1000 impressions.
  • Fill rate = Quote: Fill rate represents the percentage of ad requests that satisfy the ad requests sent by the app. It is a measure of AdMob's ability to serve ads in your app with the existing inventory.
  • You'll see eCPM everywhere, it stands for “effective cost per mile”. It's a metric indicating how effective/profitable a particular ad network has been for you so far. Don't freak out when AdMob says eCPM $0.00 when you first sign up, that just means you haven't had any clicks yet. The formula is 1000 x Cost-per-Click x Click-Through-Rate.

And we've reached the boundary of my knowledge on the topic. Hope you enjoyed the tour.

Tip #1: Filter logcat by tag “Ads” to see all your AdMob logs.

Tip #2: Search the AdMob logs for your test device ID and then plug that into the ads:testDevices attribute of your view. From what I hear the AdMob support is notoriously bad so if you get in trouble with your account, it's far from easy to get it back online.

Tip #3: eCPM is a metric of how well an ad network has been for someone. It's not a setting that you need to configure (even though it looks that way from AdMobs UI).

6. Localisation.

Not much to say here. Android warns you constantly about externalising your strings which is good. It doesn't monitor your source code though. If paranoid or you have a large team, Eclipse can warn when it finds strings without something like //$NON-NLS-1. Checkstyle et al probably have a similar feature.

Tip #1: Either put non-translatable strings into a file called donottranslate.xml (seriously – Android Lint requirement) or give them an attribute translatable="false".

Tip #2: A split to paid/free editions will introduce new strings. Consider this before sending out text for translation.

Tip #3: The marketplace description(s) will also need translation. Write and include that before sending out text for translation.

Tip #4: Your text is easier to manage when all in one place. Therefore don't include separate strings in the free/paid versions; put them all in the library project together then just reference them differently in each app. If you don't want to change the references then reference an alias and change the alias in each project. Example:

Library project:
  <string name="app_name_free">Bananas (free)</string>
  <string name="app_name_paid">Bananas (pro)</string>
  <string name="app_name">@string/app_name_paid</string>

Free project:
  <string name="app_name">@string/app_name_free</string>

Tip #5: Use the app in each language to spot-check the layout. Certain languages might need tweaking to look good. For example, a language like Japanese that doesn't use spaces might need a manual endline (\n) so that it doesn't end up with one character dangling and the word split across lines.

7. Proguard.

If you don't know what Proguard is, it's a tool that shrinks, "validates", optimises and obfuscates your binaries.

Scala on Android demands it.
Optimisation is very slow.
Obfuscation didn't seem to do very much to my app although it happily obfuscated my dependency libraries.

I managed to speed things up by having two proguard configs: dev & release. Turn off optimisation and obfuscation in the dev config and you will save a lot of time. With optimisation enabled it can take minutes at a time. I've uploaded my configs here: https://gist.github.com/japgolly/4747423

Tip #1: Reuse existing proguard configs.

Tip #2: Use separate dev & release proguard configs.

8. Signing.

Your app will need to be signed (and then zip-aligned, in that order) in order to release it. To sign it you need to have your own certificate that doesn't expire until at least 2033 (which isn't the best for security but I don't make the rules). It's a Java thing so just google keytool and you'll be done pretty quickly. You'll need to create and save 2 passwords: one for the key, one for the store. I generated mine with this command: (tip: script it or purge it from shell history when done)

keytool -genkey -v \
  -keystore <keystore filename> \
  -alias <keystore name> \
  -keyalg RSA -keysize 4096 -validity 10000 \
  -keypass 'xxxxxxxxxx' \
  -storepass 'yyyyyyyyyy'

Once you're done, make sure you backup your keys and passwords then it's time to integrate signing into your build. For a Maven project you need to weave a massive blob of shit into your poms in order to integrate signing and zip-align. They should be in a release profile and all up it comes to around 100 lines of XML (!). I don't remember how much I was able to refactor into my parent pom (as my project is multi-module) but I don't think it was all of it. I'll post all the Maven stuff shortly anyway. Most of the time spent on signing was getting Maven to work properly.

Tip #1: Either ditch Maven for something better, or make sure you copy a working Maven Android project that has signing. (I'll post my Maven stuff shortly.)

9. Build Automation.

Last but not least, we arrive at something that is actually important from day #1 of the coding period: build automation. Very important. Obviously. I used Maven and ended up creating a multi-module project which I'll post more on later. Suffice to say Maven gave me trouble, and cost me too much time. I want to switch to something else and I have my eye on SBT.

I did some tests on SBT recently and the results supported the rumours that SBT is faster than Maven. A build of a single, simple Android project took 35 sec with Maven, verses the same project at 24 sec with SBT. That's a 31% saving. A clean build of my multi-module Maven project takes 2 minutes and I wonder how fast it would be with SBT. If I assumed flat 31% saving again, that's 120 sec down to 82 sec. Nice. I didn't investigate further because it's time-consuming and because I worry about Eclipse integration. I'm sure Eclipse + SBT = happy days, but it seems that Eclipse + ADT + SBT = pain. ADT is extremely inflexible and the sbt-android plugin is way too poorly documented for an SBT noob like me. If I changed the dir & file structure to suit ADT an expert might be able to modify the sbt-android build pretty easily but I can't. Also, it practically has a seizure if you give it a full AndroidManifest as it wants to create its own. I imagine that ADT wouldn't be too happy about that. Would loooooove to be shown the error of my ignorant ways here so if anyone knows, ping me.

Another thing is that Maven is archaic and verbose. Reuse between modules is impossible in some situations; there are 6-year-old bugs and feature requests for mixins that are still open... SBT on the other hand is concise and allows as much reuse as you can shake a stick at (and I can shake a stick at A LOT of reuse).

But any further bitching about Maven is just that at this point: bitching. So I'll suck it up and declare that though sweat and toil I've come to have a great multi-module Maven Android project setup with library, free/paid projects, and free/paid (instrumented) integration test projects working together, nuances coerced, effective for dev and release builds. I'll be posting it shortly.

In Conclusion

Looking back on all these issues I can now see how it took 5 or 6 days to go from 95% dev complete to released, despite doing 0%-95% in 2 days. I'll be applying these lessons during my other Android projects and I hope it helps others (yes you!) as well.

Sunday, 18 November 2012

Maven: Now with Colour™ !

Maven.

I'm an command-line kinda guy. GUIs didn't event exist when I was a kid. When I use Maven it's often from the command-line. A curious thing happens when you run Maven from the CLI more than 3 times in a row. You slowly lose the ability to read those useful little symbols that we affectionately refer to as "the alphabet", your eyes begin to ache and you start to wonder who's really in control: you or your eyelids, the list oozes on. The reason for this is that you get a lot of information as the build runs, and more importantly it's all the same colour, usually bright white on black depending on your terminal config. Things get really hard to see and it takes a lot of precious concentration and brain-juice to decipher the results. Usually this is because you're searching for 1 particular line out of 100 or so with no real visual hints. I don't like it. My brain may well just be getting old or maybe I'm just working too much lately but I find that I don't really have much brainpower to spare, especially when you're in the middle of a problem with a large context. Don't like it.

So I wrote a script.
Get it here: https://gist.github.com/4104053

The great thing about Linux is that you can treat nearly everything as either a file (like /dev and /proc, awesome!) or a pipeline. By simply piping Maven output through a processor (while being careful not to block) you can apply colour all over the place by using some smart regex to insert strings that the terminal parses in order to allow user control over various terminal attributes, in this case: colour. (See ANSI escape codes on Wikipedia for more info.) Now I'm not the first person to take this approach, but the great thing is that because these scripts are so easy to write, anyone can create or customise one for their own personal needs.

My script uses said approach to do the following:
  • Clear the screen before running.
  • Highlight the name of each Maven phase and the plugin & goal responsible.
  • Colour the number of test passes, failures, errors.
  • Colour the test classes & methods of test failures and errors.
  • Colour Maven warnings & errors.
  • Highlight total build success / failure.
  • Remove that bunch of shit you get at the end of failed Maven builds. When Maven fails (or tests fail) it dumps a bunch of info that basically amounts to "Try using -e or -X else here's our website." Ok for beginners maybe but I don't need to see it all the time. I have "mvn -help" and Google if I need them.
Now that feature-set might not sound like much but makes a world of difference. It turns my frown upside down. Twice! (Hey...but that means-

Here are some screenshots:

Coloured Maven when things are happy


Coloured Maven when things go wrong

Thursday, 1 November 2012

Rant: Stupid Maven

I'm back in the Java world again, and already I'm annoyed. The Ruby world isn't without its problems but Ruby problems are generally much easier to solve when they do appear. You generally don't end up wanting to punch the screen and scream "give my life baaaack!" (Thor's internals get me close though.)

Today's problem: Maven.
Now, I've been a big fan and advocate of Maven for years. Therefore, upon returning to the Java world I happily sought out my good friend Maven and started using it immediately with my new project. ...Only to have problem... after problem... after problem... So now I'm going to vent.

  • TO DEPLOY to a local instance of Nexus you need to plonk big blob of shit (ie. XML) in every single project POM file. Adding <distrubutionManagement> in user profile's settings.xml doesn't work. WTF! In Maven-land, any artifact can be deployed to any repository so long as the repo configuration allows it. A beauty of Maven is the interoperability of artifacts & repositories, this is, a project that can deploy to Maven Central can also deploy to an intranet Nexus or Artifactory; no code needs to change; the protocol is standard. Forcing the storage of repo settings alongside the project's build instructions doesn't seem very bright to me. It's nice... but how to build a project should depend on where a project's gonna go after its built (especially when there's no impact on build anyway). In reality they are separate concerns but the system doesn't allow that separation. If repo info could be stored in either project or user-profile settings so that people could decide based on the project circumstances then fine, but as it stands currently, it is infuriatingly not the case and not supported.
  • SO, NOW to make things work, I need to have the same parent pom for all of my projects that I plan to deploy to my local Nexus. The alternative would be to copy & paste the same 20 lines or so of XML into each project: umm no. What's annoying about this is a) it's something else to maintain (as opposed to settings.xml which wouldn't have to be deployed or available as an artifact), b) it introduces a new dependency (literally, not runtime) to all my projects now, great; and c)  Maven poms use single-inheritence -- no mixins; to inherit from a different parent I have more stuffing around to do. Bad architecture. Annoying.
  • Parent POMs (multi-module or standalone) allow you to configure plugins, however those plugin settings aren't automatically inherited in child projects. For settings I think you need to redeclare the plugin in the child project (not 100%, I don't even care anymore). What really bugs me is that version specifications are not inherited. WTF GUYS! So if I have a multi-module project (and I tried this) where in I declare the version of a plugin in the parent pom, the child does not use the specified version of the project. OMG. I tried with <inherited>true</inherited>, I tried in both <plugins> and <pluginManagement>, it doesn't matter. Now I guess I'm supposed to declare the versions in as properties and specify plugin versions in every single child module instead? Not only am I averse to duplication for the obvious reasons, but now my pom files are all massive! Now I googled this and apparently this is deliberate because "you should always specify specific versions, inheriting versions is evil and disallowed", yada yada. Yep, well it's one thing in the context of isolated projects but in the context of multiple modules comprising a single project, it's the opposite. The modules are components that are meant to be built together to create a larger system. Declaring certain plugin versions at project-scope is entirely reasonable and has many advantages. Secondly, when you don't specify a plugin version, where does it come from? The build still works so it's coming from somewhere, right? It comes from the Maven super POM which is like a template that all poms inherit. In it, default plugin versions are specified left-right-and-centre, and they are inherited and used without explicit specification or "evil" consequences (although there are some unless the release plugin has been updated to hardcode all plugin vers in child poms during release - haven't used it in a few  years). Double standards.
  • Finally XML. Come on. It's 2012. XML has always been a tedious, ugly, unacceptably verbose format. The reasons that made it appealing in 1998 are no longer valid in my opinion. Especially not in the context of pom.xml. One the best things about Maven is that you don't have to write a bunch of crap for every project in order to get it to build in an automated fashion; Maven allows you to simply say "I'm blah v1.0 and I need v2 of X and v3.4 of Y to build. See ya!" and it takes care of the rest so long as you follow its standards. Add a few dependencies though and add a single command (argLine) to surefire (the Maven testing plugin) and you're looking at over 100 lines of XML. Writing everything manually in Ant would only come to about 80 or so and it uses XML too! There was a project called Maven Polygot a while back that was supposed to allow the pom to work in different formats but it seems dead or at least progressing slowly enough that it is moot anyway. Progress is pretty generally slow in Maven-land. Point: In this day and age I resent having to use XML almost anywhere. Have you ever filled in a form by hand, a tax return, a car rego, and seen:
    Phone Number: ________________ /Phone Number
    I haven't. I really dislike XML and I'm glad it's finally starting to trend away. Look at how concise things could be:
Rant.stop.