You need to ask for some time before you give an answer back to management. Go back and study the task:

• Write down an overall approach
• Maybe do some exploration of the code for a few hours
• Break down the task in steps in an excel (I would suggest Microsoft Project, but that may be an overkill)
• Against each task try to assign your best guess of estimate
• Add them up and add a 20-30 percent buffer and give to Management

Over time you will get better.

Update:

Well, it doesn't have to be 2 days. You basically don't give an answer right away. There is no way you can know (except for the most trivial tasks) how much effort it will take right away. If it's a large task (maybe 3-4 weeks) it will take you a couple of days to estimate it, but if its a task which is a 3-4 days, then it will only take you 3-4 hours to break it up and estimate.

estimate quickly, re-estimate regularly and most importantly keep management updated.

It's all about managing expectations, so make sure they understand the limitations of your estimate, this is key, especially for non technical project managers.

This may sound glib, but it actually does help. I have had a couple of internal customers over the years who would burst into my office with a new idea. "How long would it take to write a new feature for our big program that does [extremely vague description]?" I found that asking for more details was often futile. "Just give me a rough estimate," he or she would demand.

So I would reach into my desk drawer, and pull out a six-sided die. I would shake it up in my hand and toss it on the desk. "Four weeks," I would pronounce.

"C'mon, be serious," the customer would complain.

At that point, I would explain that without at least a coherent idea of how this new feature would work, I had absolutely no idea, and rolling the die was as good as any other method of estimating. Ultimately, they all got the hint. Most ideas went away to die. With the rest, my customer would come back with something I could work with.

In a nutshell, what I was doing was illustrating the absurdity of trying to come up with an estimate with virtually no information. A picture is worth a thousand words, as they say.

Estimating is hard. Working with NO estimates is a pipe-dream. So it's sort of a necessary evil.

• Start by making it clear that initial estimates are way off so if you come to a X Unit estimate, give a range 0.6X - 1.6X.
• Then there is the trick of doubling the estimate (however saying a ~2 year will take 4 years might be a problem)
• (There's one more involving scaling the magniture and bumping to the next higher unit. Need to refer to my book for this one.)

Next comes the part of how do I reach that estimate so that I can double/multiply? No easy answer.

• Historical trends make sense if you're doing the same kind of project with the same level of team skills.
• Next is the 'Let me steer the ship for 2-3 iterations' and then I'll get back with a better estimate.
• Finally comes the "Tell me how much it will cost.. Till then no one works on this." In this case, you have no guardian angel. Sit and break it down and then apply piecemeal guesses till you have a cumulative estimate. Then do the double-n-bump.

In my 5 years of living in the trenches.. The only thing I've seen is 'guessing by the seat of your pants' estimation... No FP.. No process.. helps in real-world constraints. Still things can improve.. for that I'd suggest getting Mike Cohn's Book 'Agile Estimating and Planning' - that's the book that is closest to us sinful programmers and reality. The rest are preachers on their high horses... Caper Jones which planet is your abode?

Nice question.. I keep asking this when every project begins and then once every week i'm doing the GRIND. Still no answer from the voices in my head.

Also, since it sounds like you don't work a purely software company, keep in mind that nowhere near 100% of developer's time is spent on coding. Make sure you put in time for planning, documentation, research, testing. These are usually the steps that managment would like to cut out of the process, but they won't be happy with the results if they do. If you get any backlash you should remind them, that A) You will end up spending time on those items anyway, whether they are budgeted are not B) You will end spending more time on maintaining and fixing bugs without proper planning. If you haven't read "The Mythical Man Month" by Frederick Brooks, I would hightly recommend it. It has a lot of good concepts and you'll probably relate more than a little to some of the scenarios. Some of the items are a little dated, but there's a lot of good info and you should be able to get one on the cheap.

Don't estimate. Instead, use a project management technique that's not reliant on estimate. Scrum, for example, stresses doing work rather than talking about doing work (ie estimates).

If you spend a day or two or 5 or whatever estimating, wouldn't you be better off having coded or designed or tested on those days, instead of spending the time "estimating"?

Double your initial estimation.

Seriously though, try to break up the task into smaller, estimate-able bits. And then double it.

I don't think there is a universal step-by-step prescription.

Every project is different. If you know your code, and you know your processes, then you know enough to do a serious estimate. Just think through all the steps, all the components, and be realistic. Don't be overly optimistic. Build in contingencies for all the places where there is potential risk. It is far better to be accurate than to be fast.

Like anything else: you get better at estimating by doing it a lot. The more you force yourself (or are forced) to make estimates, the better you'll get at doing it.

To get better at making estimates, you need to compare your previous estimates against the actual time it took and then look for patterns in the difference.

As others have suggested, the first step is to break the project down as far as you can so you're working with manageable discrete tasks. Record your estimates for each task and then, as you do the work, record the actual time it takes (including the time for all the bits you forgot about when making the estimate - they're often where the biggest problems lie).

At the end of the project, compare your two sets of figures to see where you went wrong (because, even after years of experience, we all go wrong somewhere). You can then look to improve in two ways:

• Try to spot where you're consistently underestimating or forgetting tasks and then take that learning forward to your next estimate - ie try to improve the accuracy with which you estimate each task. The more you work on similar projects each time, the easier this is to do.

• Also look for an overall pattern in the difference between the total estimate and actual time taken. If you're consistently taking, say, 50% longer than you estimated for each project, you have a guideline for increasing your future estimates. So even if the accuracy of your individual task estimates doesn't improve, you should have a more realistic overall esimate.

You can benefit from the first approach after one project - the second approach becomes more effective as you have more figures to compare.

I also find the second approach to be more helpful if you need to estimate the time other developers will take. I find it hard to estimate on behalf of another developer in a team, but I know that certain people consistently take more or less time than me on particular types of task. So rather than try to work out how long they'll take, I estimate how long I'd take and then apply an overall multiplier based on previous projects. (Ideally, of course, I'd get them to make their own estimates, but this isn't always possible.)

Ask your boss what decisions s/he is going to make based on the estimate you give.

Estimates are decision support. What most people really want are better decisions.

Maybe we can get that by improving our estimates (and there have been a lot of good suggestions given) but, let's face it, estimates are never going to be as accurate as the average PM wants, nor as low as the average salesman wants.

All estimates are ranges and probabilities (even when we're forced to give a single number), and you can provide a lot better decision support if you know what the estimate is going to be used for.

There's a lot more to be said about this, but I wanted to get the perspective out there as no one else has mentioned it.

Remember Hofstadter's Law:

It always takes longer than you expect, even when you take into account Hofstadter's Law.

It's funny and all, but I actually usually make reasonable estimates, then put some (at least) 10% on top of it and then round it all up. In occasions I was not allowed to do it, I regret to say I ended up failing miserably in keeping up to schedule.

Something I use is COCOMO. It's a technique that provides probabilistic models for software cost estimating. It's something that your PMs (if formally trained) will likely be familiar with. Instead of being concerned about hours, you simply estimate the SLOC (source lines of code) you expect to write. That right there should set off some alarms, but it's only one piece of the technique. You then plug these estimates into a model and adjust the metrics for fine tuning.

• Number of developers
• Capability of each developer
• Your work calendar (hours per week, per resource)
• Development language(s) & percentages (if using more than one)
• Familiarity with the domain (has anyone written this type of app before)?
• Management style
• Development process (Agile, waterfall, RUP, ...)
• etc.

What this whole process does is let you focus on what you're good at estimating: code. The time estimate is abstracted and encapsulated by the model itself. Each model has been constructed by analyzing 100's of real-world projects. You can even tune them yourself, by plugging in your own actuals.

I've had a lot of resistance to this method, especially when I forget to round. "Oh, right. This will take 29.02 hours." All in all, it's been the most accurate method for me. My colleagues can disagree with the usage of SLOC all they want, but they cannot debate the science that went into creating this technique.

As a sole developer, I have used it, but had problems with my ego at the beginning. I didn't want to acknowledge that my C++ skills were weaker than I thought at the time. It caused me to underestimate. After I plugged in my actual skill levels, everything lined up. Lesson learned.

Another technique I throw on top (some tools do this for you), is the three estimate technique, which is broadly applicable in all places. You take your worst case, best case and expected case then average them together. Weight your expected by a factor of four. So... "(worst + best + expected * 4) / 6" It's a moderate help for risk, but it's actually a psychological trick to get accurate estimates more quickly... in favor of your gut feeling.

Overestimate and break down the project in multiple small tasks wich can be devided in either 15 minutes, 30 minutes, 1 hour, 2 hours, 4 hours or 8 hours; If it doesn't fit in 8 hours split the task up again. This should give a fair idea of how long it should take.

Estimating software projects is a shot in the dark. Don't let anyone tell you otherwise. Project managers, especially non-technical project managers, often don't realise this.

Although there are lots of guidelines and formulas out there I've never found a better way than just taking a previous project, applying some kind of scaling factor and using that as a baseline. For example, my last project was 6 months long and my next one has simpler rules but lots more data migration plus another external interface so I reckon its about 50% harder = 9 months.

Some tips:

There are a lot of good suggestions on how to do the estimate but they all take time. To get you the time to do an estimate correctly I use this tactic.

When I’m asked for an estimate with out being given time to figure it out how long the project will take I always come back and give an estimate I know management will not like. Let’s say you think a project will take about 3 months. I will say something like 6 months to a year. They usually give me a bad look and I say, “Well, if you let me do a real estimate I can probably give you a better estimate.”

Also, NEVER say a project will take 2-3 months! You are thinking the project will take 3 months and your manager will only hear 2 months.

i'm late in the game for this question, but your comment struck a chord: you are often asked for estimates on the spot - no time to research, detail, plan, consult with the team, etc. Just a go-with-your-gut number - and probably a number that will come back to haunt you.

This happens - frequently. The trick is to trust your instincts but also compensate for your errors.

In other words, take your best guess and multiply it times some factor that helps to cancel out the lack of information, time to do a proper analysis, and eternal hubris and optimism.

For example, if your last 3 estimates were:

• 1 day
• 2 weeks
• 3 months

and the actual time was:

• 4 days
• 2 months
• 1 year

then clearly your "coefficient of optimism" is 4.

As you get better at estimating - or avoiding estimating, as the case may be - this coefficient should trend downwards. But it may go back up again for a while if you start working in a different domain or technology or team, etc. So it's important to always track it, at least mentally.

For a brand-new area with little info and unfamiliar technology in extreme circumstances - such as repairing the hyperdrive on a Hutt battle cruiser in the middle of a firefight when you can't read Huttese - it's best to employ the Scotty Rule: times two and add a few ^*

[^* a friend of mine insists that the true Montgomery Scott Rule of Estimation is to take your best estimate, round up, multiply it by 10, and change to the next higher order of units. So if you think a task will take 2.5 hours, you round up to 3 hours, multiply by 10 to get 30 hours, and change the units from hours to days to get the final estimate of 30 days.]

I would recommend to split task as much as possible before performing any evaluation, this exercise only will force you to have a deep understanding of what you have to do, and how you plan to do it. Once this is done, you increase your chances of comparing the subtask to something you have already done and have something closer to reality. If the specification of what you have to do is unclear, you will also find out at this moment, which is a good thing.

Tasks over 40h can hardly be evaluated with precision, if your project manager is good, he'll know that! It's ok to be wrong, experience is the only real thing that will make a real difference in your evaluations (and in the end, will only help you be closer to a correct answer, not perfect).

I also had a very nice experience with poker planning, the more my team uses it, the better their evaluations (plus it's fun!) but it takes more than 1 person which can't help you much.

There's usually an asymmetric penalty function here, for the manager and the dev team. That's actually part of the logic of padding.

Late: If people tell their bosses to expect Tsched, and instead it is longer than Tsched, then schedules get bumped and people get faulted for various things and in the worst cases sales gets missed or customers go away or projects are re-organized or trash-canned.

Early: If the true T is less than Tsched, then hey, pat the dev team on the back and get busy on the next project. Or -- heaven forbid -- refactor and improve the code quality (no, don't do that... do the next project. workworkwork).

You should listen to the stackoverflow podcast and hear Jeff Atwood's estimation process for this site you are on right now. He was a little too aggressive and Joel called him out on it.

These are good points to consider, but you'll need to weed out things that don't work or apply. Revisit the suggestions after you do more estimates, and see if anything stands out as more applicable after using these suggestions.

Keeping track of what you estimated and then how long it actually took is a GREAT idea, and one too often not followed up on, because you need to move on to the next "fire". But, you can only get better at estimating if you spend the time to determine how your estimate was off, and figure out what to change for the next time.

You also mentioned you are working with existing code. As you make more changes to this, I would suggest increasing the time needed to test. You need to make sure the change works, and you need to make sure the other functionality of the code still works. If your tests are somehow automated, then you may need to change the to test this change to the software. If your tests are manual then you need to be sure to test what you changed (adding or changing how you tested previously).

Good luck.

You can try to break it down in smaller tasks, and estimate each task. For example, you could break it into something like:

1. Install SharePoint
2. Create proof of concept web part.
3. Deploy custom code into SharePoint.

These are tasks that you should be able to find tutorials for, and by reading them, you may be able to grasp how long it will take (hint: installing SharePoint is long!).

Besides this, I agree that it's hard to estimate tasks that you haven't done before, so try to break it down to tasks that are small and comparable (in scope) to things you've done in the past.

If someone else in the company has done similar stuff yell over the cube wall and see what they think. Compare that to how long it actually takes you, then when you get another task ask the same developer how long it would take them. Multiply that result by the difference you got doing the initial task. There's really no good way to estimate new tasks with out a historical baseline but at least this way will show you made an effort to base your estimate on some kind of fact.

As a developer who became a project manager I can tell you that this is not easy. The key point is to break the task down into smaller and smaller chunks, estimate each one then add your estimates together. Obviously this may take some time so speak to your PM about how long you have to produce the estimate. Also any good PM will know that an estimate is just that, so keep them informed with your progress, remember ultimately you are all part of the same team.

Hope that helps,

Dan.

The important thing is not giving an estimate that "sounds" OK for your Boss because he's expecting something, just really estimate what you think. If he questions your estimate, just say that you'll keep him updated from time to time and remember him that you've never work with the technology.

Also, I always make two estimates, the pessimistic and optimistic scenario. I ADD 10 to 20% at the total for tests and debugging (normally, nobody will ask you to remove the debugging time).

Estimate on the estimate.

In many cases coming up with the estimate for a project represents a lot of the work involved in the project. If you can establish the trust of your clients, you can charge a day-rate to develop an estimate. This has a number of benefits for you and for them.

1. The estimate will be more accurate
2. You'll know what you're about to do and they'll know what help you need
3. You can walk away without hard feelings if the project looks too expensive

The standard timeframe for planning on construction projects is 10% of the total timeline for the project. No reason to throw out hundreds of years' learning for a few transistors.

EDIT : For smaller projects this phase is usually where the needs assessment and feasibility study get done for free by a hungry developer .... most of us have been there ....

Based on your question it is obvious your management has as little experience with managing software systems as you admitted to yourself. What's needed is retraining management, but that will take time. My advice to you is this:

• If management demands an answer on the spot, tell them it will take X amount of time to give you a more firm estimate, but my soft estimate is .... Give a wide range, and if they ask for a narrower range inform them that you need time to do the proper estimate. NOTE: always provide at least a day so that you can come up with the firm estimate.
• Always state your assumptions with your estimates. I.e. assuming I have no interruptions, I should be able to do this feature in X amount of time.
• Give your firm estimate to the manager after you have enough time--even if they didn't ask for it. They'll start to get the picture that the firm estimates are more valuable.
• Keep track of your estimate and the actual time you spent. This will help you know if you need to adjust your estimation process a bit.

As always, an estimate is a projection, not a hard and fast rule. What this means is that when it looks like you are not going to meet your projection, you need to let someone know. That will let your management chain make adjustments before they commit to the wrong people an unrealistic date.

No matter how good you think you or your team is, always overestimate.