OpenStack horizon : fix the Instance & Volumes page

In the horizon OpenStack dashboard, the display of the Instances & Volumes page fails for every account with administrative permissions, after a volume has been created and attached to an instance. While there are workarounds (such as using an unprivileged account to display the same page ), it affects almost all admin users deploying the current release of horizon in Essex. The bug was fixed and uploaded to Debian GNU/Linux sid. It was unblocked and will be in wheezy by the end of september 2012.

diagnostic

By default horizon is silent and the error only shows as a code 500. A more verbose error message can be displayed by editing /etc/openstack-dashboard/local_settings.py on the machine running horizon and set:

DEBUG = True

The problem was reproduced with a minimal number of steps to help writing the bug report. The error page shows a full stack trace, including the line raising the error.

bug report

The bug was filled against horizon upstream and in Debian GNU/Linux.

workaround

While a patch is worked on and until it is approved by the OpenStack Stable Branch Maintainers, the following workaround was proposed.

--- views.py~   2012-09-20 17:55:03.000000000 +0200
+++ views.py    2012-09-20 20:24:45.000000000 +0200
@@ -71,7 +71,13 @@
                                     self._get_instances()])
             for volume in volumes:
                 for att in volume.attachments:
-                    att['instance'] = instances[att['server_id']]
+                    try:
+                        att['instance'] = instances[att['server_id']]
+                    except:
+                        class FakeInstance:
+                            name = 'UNKNOWN'
+                        att['instance'] = FakeInstance()
+
         except novaclient_exceptions.ClientException, e:
             volumes = []
             LOG.exception("ClientException in volume index")

When applied, the page no longer crashes and displays UNKNOWN when the name of the instance to which the device is attached cannot be retrieved.

Debian GNU/Linux bug fix

Priority was given to fixing the bug for the Debian GNU/Linux wheezy release. A clean patch was produced included and advocated for inclusion in wheezy. The associated tests were checked to cover the code by running ./run_tests.sh –coverage at the root of the horizon source tree and browsing the HTML report ( the fix is around reports/horizon_dashboards_nova_instances_and_volumes_views.html#n73 ). The packaging repository was tagged with revision 2012.1.1-6. Before being uploaded, the package was installed and tested the.re.
An unblock request was sent and accepted within 24h.

Upstream fix

Fixing the bug upstream is a longer process that requires to obey the Stable Branch maintainance process. It may take longer than for a simple fix because the code was refactored in the master branch. The bug is no longer there, as a side effect of the refactoring and not because of a patch that could be backported. A detailed explanation of the problem and the fix was written for discussion.

Description: Fixes a Keyerror when displaying Instances & Volumes
 .
 bug 1053488 prevents the display of the Instances & Volumes page for
 every account with administrative permissions, once a volume has been
 created and attached to an instance. While there are workarounds (
 such as using an unprivileged account to display the same page ), it
 affects almost all admin users deploying the current release of
 horizon in Essex.
 .
 The source of the problem is that the relevant portion of code loops
 over all existing volumes while it only has access to the instances
 that are owned by the current tenant. As a consequence, it fails to
 find the instance to which a volume is attached when it does not
 belong to the current tenant.
 .
 A possible fix would be to change the behaviour of the volume list
 API so that it only returns the volumes of the current tenant even
 when the user has administrative rights. However, this would be a
 user visible change that may have side effects beyond the current
 bug.
 .
 The proposed patch catches the lookup error when the instance is not
 found for a given volume and creates a fake instance object which
 will only be used to display the name "UNKNOWN".
 .
 The associated test re-creates the conditions and derives from
 the class that will give administrative permissions to the test
 user. However, since the data is created from fixed data instead of
 being actually retrieved from the API, this derivation is only
 included to illustrate the purpose of the test.