I’ve done this before, and this version has a significant flaw: it is not exception-safe. The destructor should not read as it does:

    ~finally() {
        finallyClause();
    }

This allows exceptions thrown by finallyClause() to propagate out of the destructor, which, if an exception is already being thrown, will result in a call to std::terminate(), causing the application to die horrifically. Because C++ kinda-should’ve-but-really-doesn’t have checked exceptions, we must simply discard them:

    ~finally() {
        try { finallyClause(); } catch (...) {}
    }

You can also avoid problems with exceptions and std::function’s dynamic allocation by using a template instead:

    template<class F>
    class finally_type {
    public:

        explicit finally_type(F f) : function(f) {}
        ~finally_type() { try { function(); } catch (...) {} }

    private:

        finally_type(const finally_type&);
        finally_type& operator=(const finally_type&);
        F function;

    };

    template<class F>
    finally_type<F> finally(F f) { return finally_type<F>(f); }

This can be used like so:

    void test() {
        int* i = new int;
        auto cleanup = finally([i]() { delete i; });
        may_throw();
    }

Finally (har!) the equivalent idiom in C++03:

    void test() {

        int* i = new int;

        struct finally {
            finally(int* i) : i(i) {}
            ~finally() { delete i; }
            int* i;
        } cleanup(i);

        may_throw();

    }