Tag Archives: howto

Are you getting errors building Amazon lambda functions? Don’t fret I got you!

aws lambda python

Why does Amazon make lambda functions so hard to create? Well my guess is that when you live at the bleeding edge you should expect to get scrapes!!

Everybody is trying to build lambda functions these days. And it’s no wonder. Once you get them running, Amazon takes care of all the infrastructure drudge work! So cool.

Join 32,000 others and follow Sean Hull on twitter @hullsean.

I’ve been spending some time trying to get answers out of AWS support, and let me tell you it’s no fun. Yes all this stuff is new technology, and nobody has expertise in it in the way say you might in Linux or Oracle or another technology that’s been around for a decade.

Still you’d hope the techs would have some clue. In the end it was a slog dealing with support, and I think I was the one teaching them!

I did find Matt Perry’s howto which is pretty good.

Hopefully my own notes can help someone, so read on!

1. No lambda_function?

The very first issue you’re gonna run into is if you name the file incorrectly, you get this error:


Unable to import module 'lambda_function': No module named lambda_function

If you name the function incorrectly you get this error:


Handler 'handler' missing on module 'lambda_function_file': 'module' object has no attribute 'handler'

On the dashboard, make sure the handler field is entered as function_filename.actual_function_name and make sure they match up in your deployment package.

If only the messages were a bit more instructive that would have been a simpler step, but oh well!

Also: Is Amazon too big to fail?

2. No module named MySQLdb

This is a very tricky one. I mean after all you just spent all this time building your deployment package specifically for lambda, so what gives??


"Unable to import module 'lambda_function': No module named MySQLdb"

Turns out when you use a virtualenv, files will be installed into proj/lib/python2.7/site-packages/ or lib64. However Lambda wants them in the root proj/ directory! So move them there. I know I know. Weird, but that’s what they want.

Related: When hosting on Amazon turns bloodsport

3. Can’t find libmysqlclient

If you’re using the MySQLdb library like I was, you’ll eventually bump into this error:


Unable to import module 'lambda_function': libmysqlclient.so.18: cannot open shared object file: No such file or directory

Turns out that /usr/lib/libmysqlclient.so.18 needs to be COPIED from /usr/lib. Don’t do “mv” or your system won’t have the mysql lib anymore!

Related: Are SQL databases dead?

4. Use the Amazon Lambda environment

One thing the support pointed out is that AWS as *supported images* for lambda development.

After all the errors above were resolved, it’s not clear to me that the supported AMI’s are truly required. However if you’re hitting intractable problems building a properly lambda deploy, you might wanna look at building one of these boxes.

Read: Why dropbox didn’t have to fail

5. Build your lambda deploy package

Now let’s roll it all together. Here’s are all the steps to build your deploy package.


- SSH to the instance
- mkdir test
- virtualenv test
- source proj/bin/activate
- sudo yum groupinstall 'Development Tools'
- sudo yum install mysql
- sudo yum install mysql-devel
- pip install MySQL-python
- cd test
- emacs -nw lamdba_function.py
- add your code to that file
- save the lambda_function.py
- mv proj/lib/python2.7/site-packages/* proj/
- mv proj/lib64/python2.7/site-packages/* proj/
- rm -rf proj/lib (don't need dist-packages in the deploy pkg)
- rm -rf proj/lib64 (don't need dist-packages
- zip -r proj.zip *

Also: How to hire a developer that doesn’t suck?

6. Upload your code

Uploading your code via the AWS dashboard is fine when you’re first testing things. But after a while it’ll get tiring going in the front door.

Create a new lambda function by specifying the basics as follows:


aws lambda create-function \
--function-name testfunc1 \
--runtime python2.7 \
--role arn:aws:iam::996225510001:role/lambda_basic_execution \
--handler lambda_function_file.handler_name \
--zip-file file://proj.zip

And when you want to update your function, do the following:


aws lambda update-function-code \
--function-name testfunc1 \
--zip-file file://proj.zip

Also: How to deploy on EC2 with Vagrant

Good luck with lambda. Once you get past Amazon’s weak documentation it’s pretty cool to be in a serverless computing environment. Happy deploying!

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Best Howto posts on Scalable Startups

vagrant logo

Join 28,000 others and follow Sean Hull on twitter @hullsean.

MySQL slow query on RDS

If you run MySQL as your backend datastore is there one thing you can do to improve performance across the application?.

Those SQL queries are surely key. And the quickest way to find the culprits is to regularly analyze your log. I’ve put together a howto & script for doing this on Amazon RDS.

Automate mysql slow query analysis on Amazon RDS

Vagrant & Amazon EC2

Among the automation tools that are getting a lot of attention these days are Chef, puppet & ansible for devops, plus Docker & Vagrant.

Can we use vagrant to spinup EC2 instances? As it turns out yes. And it can be a great way to automate dev environments and works in conjunction with docker.

How to deploy on amazon EC2 with vagrant

Cache websites for speed

Is Fred Wilson right that speed is an essential feature? We certainly think so.

And besides tweaking & tuning the database, the next best way is caching. You cache objects at the browser, add a page cache and memcache, redis or elasticache. Here’s our howto.

5 tips to cache websites & boost speed

DB Change Management

Everyone uses version control for application code, whether it’s PHP, Ruby or Node.js. But are you using it for database changes?

DDL, those statements that create objects should also be included in version control. But how to do it properly? Database change management is one part art, but there are some helpful tools to get you on the right track.

With some luck you’ll be able to roll forward & backward to versions of your database schema just as easily as you can versions of your software.

5 tips better db change management

MySQL Scalability

MySQL is the big bad beast that still hobbles a lot of site performance. Here are some key tips, narrowed down to just the essentials.

5 ways to boost mysql scalability

Cloud Scalability

The cloud enables scalability, but is it ready out of the box? There are some key things to remember on your road to high scalability in the cloud.

3 ways to boost cloud scalability

Fortify MySQL Replication

MySQL replication is pretty awesome for what it is. Still there are gotchas & potholes. Here’s our guide to smooth sailing.

5 ways fortify mysql replication

MySQL replica with Hotbackup

If you’re building your own MySQL instances on EC2, you’ll also build your own replicas. Luckily there are some great tools that make this reliable & smooth. Install percona’s hotbackup tool & you’re off to the races.

Easier mysql replication using hotbackups

MySQL Backups

If you’ve forgotten all about backups since your cloud or managed solution does all that for you, think again! There are still things you should do in addition. At the very least run a fire drill & find out if all the parts are there for rebuild.

10 things to remember with mysql backups

Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters

Consulting essentials: Building your business

In the last two posts on how to build a successful consulting business I shared advice and tips on closing deals and managing and completing your engagements.

This post will look at where to focus your efforts in order to sustain your consulting business, and build skills.

Focus on your subject matter expertise

Being a subject matter expert takes years of education, and professional experience to build. It’s your most valuable asset. Build it, and use it. This is not to say there isn’t great value in being a generalist as I’ve emphasized in blog postings. But don’t get distracted trying to do things others can do better. Web-based newsletter managers can do it better than you can, professional invoicing solutions as well. Though you might be able to do a bit of HTML and graphic design, if that’s not your expertise, hire someone to do your website in WordPress. You’ll save time and money in the long run, and their professional experience will surely include a few surprising nuggets of value that you wouldn’t have known yourself.

Manage your business with online tools

There are a myriad of online tools to help you run your business better.

Google Analytics – Get insights on what parts of your site your customers find valuable. Learn to convert those visitors with downloads, newsletter signups and ebooks for sale.

Freshbooks, Zoho or Paymo – There are many online invoicing solutions, so best to evaluate them for your specific needs. The point is, using a solution will save you money in the end, and make you more professional in the process. It’ll help you track monthly & yearly, send reminders, and summarize things for you all in one place.

Mailchimp – Newsletter creation can’t get any more fun than with mailchimp. With endless video howtos, documents, and great support, plus templates for your newsletter that come in every shape and size, these guys have really thought of everything. What’s more the graphs and campaign reports will give you insight into what topics are popular, and what resonates with your audience.

Google Docs and Calendars allow you to share your schedule and work with a small team. So whether it’s communicating things with your clients, or your subcontractors, these tools are essential.

Dropbox – Another indispensable tool for collaboration, share documents easily across the web.

Don’t Be a Commodity

If your prospect is asking you for a resume, and comparing you to some outsourced resources provide, you’re not going to get very far as a consultant. Don’t be a generic resource that sits at the keyboard and churns out code.

What’s the alternative? Learn about business, talk to business stakeholders, listen to their concerns, and learn to speak their language. Use stories, and analogies. Reach across the table to speak in terms that everyone can understand. Your value is in symplifying complex technology.

Be Hungry & Don’t Give Up!

Being hungry means continuing to improve on your weaknesses, experimenting with different rates until you match the market demand, managing multiple projects and always networking widely. Spend less than you make and build up six to twelve months cushion. This will allow you much more freedom to take only the projects you want. Learn to pick and choose.

Further Reading…

There are lots of other areas you’ll lean about, running a successful consulting business. Here are a few:

  • managing cash flow
  • paying quarterly & yearly corp taxes
  • paying subcontractors
  • managing & accounting for expenses