Posted on by

***Another Update: I’ve created a new post with a more complex animation and have also included SVG into the mix: http://www.blackcj.com/blog/2010/09/22/canvas-flash-and-svg-mobile-comparison/ ***

***Update: I’m already three steps ahead of this article. I have already received / analyzed the optimized JS in a later post. Please read all four articles on this topic before replying or your comment will be filtered. Thanks!***

Anybody notice a re-occurring theme here? Canvas performance on the new iPod Touch and iPhone 4G is laughable at 22FPS rendering simple animation. At least the Nexus One can muster up a decent 40FPS for basic canvas rendering. Flash Player 10.1 on the other hand, blows HTML5 out of the water running at 57FPS on the Nexus One! Let’s not forget to mention that HTML5 consumed TWICE the battery life as Flash for these tests on the Android. Unfortunately, Steve doesn’t want me to know the exact battery life on my iPod Touch so it wasn’t included in that part of the testing.

All of these test were done by benchmarking in browser performance. Native applications using AIR for Android yields even better performance but we’ll leave that for the day when HTML5 can export to an APK (native Android app).

Let’s take a peek at some live footage:

Frame rate comparison:

Keep in mind Flash Player 10.1 is relatively new. If Flash had been included with the original iPhone, you would have hated it. The new version, however, kicks ass. Would love to see some hacked iPhone Flash benchmarks to fill in the missing pieces on the above graph. If the iOS can’t render a little ball, HTML5 is going to be a hard sell for iDevices.

Why do we need 57FPS? Most web content targets 30FPS.The human eye really can’t differentiate after about 30FPS. Well, this is a single object rendered on the screen. Add some game logic, additional balls maybe a graphic and you’ll quickly find the 24FPS on the iPod dip down below 15. Having extra CPU cycles to do other things allows developers to target 30FPS when building games and animations.

Battery consumption after 10 minutes:

***Addition: The graph above is in percentage of batter drained after 10 minutes of animation during airplane mode (no cell or internet traffic to get in the way). A bit more crude than the first graph but I really didn’t want to sit around all afternoon waiting for my battery to drain. I’ll try to post some more battery data soon. The most important note is that Flash didn’t consume more battery than HTML5, which was counter to my initial hypothesis.***

This graph explains itself. Canvas consumes more battery power than Flash. It’s amazing that Flash can render something at 57FPS in half the battery power than it takes canvas to run at 40FPS.

Try it out:
HTML5 (JavaScript):
http://www.blackcj.com/blog/wp-content/jsdemos/BallBounce/

Flash (ActionScript):
http://www.blackcj.com/blog/wp-content/swfs/BallBounce/

Conclusion:
HTML5 will sit side by side with Flash. Gradient fonts, drop shadows, basic video and simple transitions are probably better suited for HTML5. When it comes to rendering display objects, animation and digital rights management for video, it would be silly not to use Flash. But wait Chris, HTML5 performance is going to get better! I’ll believe it when I see it. Adobe continues to raise the bar and it will be an uphill battle to catch up.

Graphs created using this tool:
http://nces.ed.gov/nceskids/createAgraph/

Afterthought:
I’m actually quite satisfied with the HTML5 canvas for Android. It will probably work for what I was planning to do. After the iOS exporter in CS5 failed to produce quality animation results, I was hoping for more from HTML5. Basically, I feel stuck having to code in Objective-C just to hit the iOS folks out there. This essentially ruined some of my plans for creating a game framework that runs Box2D cross platform. It basically made my purchase of an iPod Touch rather worthless. Unless Apple steps up I’m going to have to exclude iOS devices from my demos or re-code everything in a third language. The whole point is not to have to re-create the same code for every device in a different language.

65 thoughts on “Flash Outperforms HTML5 on Mobile Devices”

  1. On my HTC Evo I am getting ~30 fps on the flash demo, and ~40 with the JS demo. When I zoom both flash and JS freeze. Definitely not true across all mobile devices…

  2. Thanks for posting the specs Ross. What version of Flash Player and Android OS are you using? I’d be interested to hear what other Android users are getting for frame rates?

  6. Doing animation using CSS3 Animations/Transforms/Transitions (including 3D) is GPU accelerated on the iPhone and iPad and is very fast. Doing JS + Canvas or JS + SVG animations are quite slow as you’ve found and Apple needs to do something about this for iOS.

  8. Intersting post, and while I do think performance is important I think the most important point you make is that this is really all about having MORE tools in our toolbox and not about some kind of silly “deathmatch”.

    That being said I looked through your Javascript noticed a few things:

    First, you’re doing regex in your core game loop, which seems a bit silly and is definitely unnecessary (and is probably hurting JSs #s at least some).

    Second, Flash uses the concept of dirty rectangles when blitting to the screen, and with your example at least, it would be trivial to implement the same concept.

    Finally, for what you’re doing it would probably be much faster to blit into place using an off screen image/canvas and drawImage (again, Flash uses a similar technique with it’s cacheAsBitmap option) rather than using drawArc. This is just a guess, but it’s how I was able to get the Endless Mural (http://www.endlessmural.com) as performant as it is.

    If I get some time this weekend I’ll try to make those changes myself and see how much of a difference they make. Again, good work!

  12. Also, with some very minor modifications to the drawing routine (so it only updates the stuff that needs updating), I was able to double the framerate on my iPod Touch (from ~8 FPS to ~16 FPS). See: http://wolever.net/BallBounce.html

    (of course, the new version isn’t nearly as simple as the original, but that doesn’t appear to be the point of this article)

  14. HTC EVO with 2.2, Flash 10.1 (recently updated).

    JS: 35-40 fps
    AS: 30 fps

    I’m not sure why it’s working out like this on this particular device, you’d think it would be pretty snappy given the EVO specs.

  17. Thanks to David Wolever for optimizing the JS code! This runs significantly better on my iPod Touch and Android. That’s very interesting that the volume of space being cleared on a canvas has such a large impact on the performance. I’m going to try scaling that approach over the weekend with a couple of other samples. I’d be interested to see any other JS or CSS3 approaches to this demo. Any other takers?

  18. David’s demo is really cool and outside the box, however, it doesn’t really appear to scale. I can get great frame rate with a single ball but with multiple objects it drops right back down. Flash remains unchanged with multiple display objects. Any other thoughts on canvas optimization? Or will I have to wait for GPU acceleration?

  19. I’m not sure that the performance of Html 5 is being measured here, it is merely a measure of the performance capabilities of the browser rendering the Html 5.

    Internet Explorer 9, released in beta this week, uses the multi-core CPU and GPU of the PC to achieve amazing performance of Html 5 using hardware acceleration.

    While mobile browsers will take some time to apply the same capabilities, it isn’t really the performance of Html 5 that you are measuring; just the performance of the given browser.

    Silverlight also uses hardware acceleration and is the technology used to develop mobile applications for Windows Phone 7. XNA on the other hand is used on Windows Phone 7 for developing games.

    In fairness, Flash also uses hardware acceleration today also.

    When making assertions about one technology vs. another, lets simply try and know what exactly is being measured.

  20. Great Demo. Curious to see what the results would look like using some kind of cloud computing app for either iphone or the android. As for me, I’ve had the iPhone for about a year, and have had an app called Always on PC for about a week and a half. They just came out with some upgrades on the product, and the flash player for firefox is what made me pull the trigger on it. Not bad for the price actually. Here’s a link to the demo I saw- iPhone Firefox Flash Player

  21. This demo does not use the GPU acceleration option for Flash, cause that just wouldn’t be fair to compare against 😉 I agree that HTML5 depends on the browser for performance but that’s the catch, I want my interactive content to perform on all browsers. The IE9 demos are not compatible with most other modern browsers, which sucks. Continuously playing a game of catch up is not fun. That being said, using these same demos, I see better performance with Flash on IE9 (no GPU acceleration) than I do with the HTML5 version. The bottom line is that I need performance out of new technologies that match that of existing technologies. I’m quickly growing tired of coding down to new technologies. I need code to work in both HTML5 and Flash, not one or the other. When I can build Flash in a quarter of the time that outperforms the same JavaScript, I begin to wonder where we should draw the line.

  22. For mine the most important part of your tests is the battery consumption. Not sure how authoritative or accurate your figures are, but if they’re correct, wtf are Apple resisting having Flash on the iPhone for again…?

  24. Pingback: John Nack on Adobe : Flash runs faster, more efficiently than HTML5 on mobile

  25. Hey James, thanks for the code. When building the demos, I wanted the code to be as close as possible. I built the JS version first with what I believe to be the most optimized code for canvas. Didn’t see any documentation on moving an object at runtime without redrawing. The AS version was built to match as closely as possible, not necessarily the way I would build AS code.

  26. There’s also a fill of the green boundary done on each iteration of the loop, which is totally unnecessary. Moving that to the initialization routine, taking out the regex parsing, and using Dave’s changes, I got 38 FPS on my 3Gs.

  27. Hey John B., the changes that Dave made are only applicable for a single object and are not scalable. This is my simplest demo and the code in AS and JS is the same. Both are doing the same draw cycles and both have minor inefficiencies. I have Box2D demos and more complex rendering code which is yielding the same results, actually they are showing an even a larger gap in performance. Comparing web technologies is very hard and I’ll probably never land on something both parties are completely happy with.

    Overall, JavaScript compared to Flash tends to eat up more of my CPU and feel much more sluggish across the web on both my PC and mobile devices. I expect this to change into the future but for now, I’m not satisfied with the performance. This is a call to all browsers to get this stuff to work as advertised. Even IE9 beta has issues rendering Box2D with canvas compared to Flash.

  28. Adobe will resume the work in the iOS exporter now that apple removed the restrictions. Just hope they make it faster!

  29. Pingback: Chris Black | blackcj.com: More Battery Testing with Flash for Mobile
  30. Pingback: HTML 5 vs Flash sur l’iPhone 4 : Test comparatif « Blog de Benoit Letondor
  31. Pingback: Mylenium’s Blog » The Benchmarking Trap
  32. Pingback: Swf.hu 2.0 – flash és webfejlesztés » Heti link gyűjtemény 2

  33. @blackcj.Thank you so much for taking your time to do these tests and post the results. very informative.

    @doug holland re: “…, it is merely a measure of the performance capabilities of the browser rendering the Html 5.”

    yes, and clearly HTML5 on apple’s mobile devices in apple’s browser is quite pitiful. Back when Jobs made his decree, yes Flash on mobile was really sort of a theory and there wasn’t a decent version of the player available. Adobe has clearly stepped up to the plate and is now proving that Flash on mobile devices is not only viable but in many cases better than html5.

    it will be very interesting to see how this plays out. Thx to everyone for posting their results.

  34. I wonder what performance would be like if you recoded this for iOS using SVG? SVG seems like it would be better suited to stuff like bouncing ball demo’s.

  35. How about the principle of a plug-in-free web? Stop making people download and maintain additional vectors for attacks on users’ systems. A truly open web would be one free of browser plugins.

  36. If you want the equivalent of using Flash in the natural HTML5 way, you should be using SVG. It’s a more natural fit for this kind of thing (moving objects around) than a dumb pixel-matrix like Canvas, and it (probably not unrelated!) tends to perform much better on Webkit for these sorts of things.

    I did a very simple port of BallBounce.htm to SVG, opened it on my iPod (4gen), and I’m seeing 55-57 FPS, i.e., more than double the frame rate of Canvas.

    According to your graph, I’m guessing the iPod CPU is a bit slower than the N1 CPU, yet I matched the Flash FPS exactly using only HTML5. So I’d say HTML5 is *faster* than Flash, not slower.

    “Anybody notice a re-occurring theme here?”

    Yes, that for any conclusion it’s apparently possible to come up with a completely contrived test to demonstrate its truth. Here, you didn’t compare HTML5 against Flash: you compared one tiny piece of HTML5, Canvas, against Flash.

  38. I’ll try posting this again as it seems to have been spam filtered.

    If you fix the code in the HTML5 version to not erase the background twice but simply blank out the moving ball the HTML5 version runs significantly faster than Flash. Since Flash has this, and better, tricks up its sleeve, the question is why is Flash so poor even at this?

  40. Take into account that the iphone has about twice the resolution = twice as many pixels to fill. This can be a real killer even on PCs.

    I’m also wondering if the setInterval can have any impact on the result, how does it behave if you get behind 2 intervals, will the browser try to catch up or skip the next step? It would be interesting to see a version with setTimeout in the end of the loop instead of a fixed setInterval.

    Would it be possible for you to show a demo of a more complex rendering, more object, transformed images etc. Is it the javascript or the actual rendering that is the limit?
    I don’t have any smartphone to try on yet.

  41. Thanks for this test! It is interesting to see that on my Mac the Flash version consumes significantly more processor capacity than the Javascript version. (40% of one core vs. 30% of one core, using Safari 5.0.2, OS 10.6.4, MacBook 2Ghz, onboard graphics, external display).

  42. I have to second the comments above: this is just really unoptimal Javascript code. As Tim said, this should have used SVG instead of Canvas, as it’s straight vector graphics. Even if you needed a mixture, you can overlay SVG and canvas to mix vector and pixel graphics together as needed.

    Even on the assumption that you absolutely, positively, needed to do this in a Canvas (bitmap) instead of SVG (vector), the code seems incongruent. Why are you using regexps to calculate and display framerate? Is the fact that the HTML5 version displaying hundredths of seconds instead of just seconds significant? Is the Flash version updating the framerate calculations every frame, or just periodically (harder to tell since it isn’t displaying the hundredths)?

    I think the Flash version has very different framerate calculation algorithm. If I open it in a tab on Safari, when I switch to the Flash tab the framerate shows as somewhere between 4-30, then a second later it is in the high-50s, then a second later it is 60. If it were updating on every frame like the Javascript version, I’d expect the numbers to be changing very rapidly during this phase, ramping from 4 to 60 instead of “jumping” there in 2-3 steps.

    On multiple balls, did you post that source anywhere? I’m not sure what you did, but it seems the wolever.net optimizations should work just as well for 10 balls as for 1. He didn’t do anything to the drawing logic, just pulled the framerate calculation out. So, not sure what to make of the claim of that “not scaling”, as it seems like it would (and past experience with Javascript tells me it would as well).

    Finally, have we seen it with the arc code pulled out of the loop (ie, make a sprite and blit the sprite onto the screen instead of drawing it every time)? This would be how you would use Canvas.

    I remember throwing together a Basic application to draw stuff using the old BASIC “canvas” type graphics primitives. I could do some pretty fancy stuff, but it was slow as all get-out, running 2-3 seconds per frame. Meanwhile, “real” programs were able to get 20-30 frames per second on the screen (this was very-early-90’s, so that was impressive) of much more complex shapes. Now, BASIC is slow, but was it 60-90x as slow as C or even assembly for this? No. I was just using it all wrong. Looking at this code reminds me of that.

  43. For those of you commenting on the frame rate counter, I’ve pulled it out and gotten the same visual results on mobile devices and the same processor consumption from desktop devices. I would be very interested to see an SVG implementation. Could someone please send over some code?

    As for optimizations, I had two great submissions and have re-posted on the results of those submissions. Please see this post:
    http://www.blackcj.com/blog/2010/09/20/optimizing-javascript-with-canvas-for-mobile-devices/

  45. “carl schooff
    September 19th, 2010

    yes, and clearly HTML5 on apple’s mobile devices in apple’s browser is quite pitiful.”

    Oh, damn! How terrible!

    Oh. Wait. A simple HTML 5 tweak runs the framerate on my iPhone 4 to 47 FPS. But hey, never blame bad code when you can just talk shit about Apple.

  46. Please submit performance optimizations after reading my follow up blog post:

    http://www.blackcj.com/blog/2010/09/20/optimizing-javascript-with-canvas-for-mobile-devices/

    So far, many people seem to be missing the point. Both demos redraw the entire screen. Submitting a demo that only re-draws part of the screen is not scalable. It only works for this particular demo. I’m trying to find a scalable solution and only one person has come close to creating that. I’ll create another demo to articulate this better. Thanks!

  50. My fully-updated N900 with Flash 9 (not ‘lite’) says I need to upgrade to Flash 10 to see your demo; afaik there is no Flash 10 for this (released in 2010) device.

    Your JS bouncing ball gets 25fps; the one at fatgr.in/jstest.html gets 65+

    Requiring the latest version of flash 10 for this test is beautiful :)

Comments are closed.