Our Quality and Transparency Promise

No matter where you are in the world, we know that the number one requirement for engineering software is trust. Here at ClearCalcs, we have implemented a variety of systems and procedures that promote transparency, ensure the accuracy of our calculations, and make us accountable to you at all times (it's also why 'Clear' is in our name). We encourage you to read below and learn why exactly we believe you can trust our platform.

If you have any questions or comments about anything you read here, or any comment or bug report about anything in our platform, feel free to contact us via either the "Help" button in the lower right of your screen, or via email at hello@clearcalcs.com.

Our Promise To You

Full Transparency

We believe that the best proof of our calculations is in letting you see and scrutinise every step. We never want any part of our platform to be a "black box", so we promise that all of our calculators will always show you every formula along with its specific reference in the appropriate code or standard. If for some reason we can't show a formula or if it's not practical to do so, such as when we determine beam demands using our finite element analysis backend, we will still show you all of the data, and we will use open-source modules whenever possible, so that everything we do can still be publicly checked. When we make changes or fix problems, we will record it in our fully-accessible changelogs, and when you edit your calculations, we'll record that in the calculation's audit log.

Ruthless Checking

Transparency lets us prove our accuracy to you, but before we even release them, we will ensure that our calculators are accurate for every design scenario imaginable. Even the smallest change within our platform will be run through millions of automated tests and peer reviewed by at least one other team member. When we create any new calculators or perform any major changes, we will additionally have the calculator independently checked by experts in the relevant field and scrutinised by users while in "beta" mode.

Safety First

Even with the most careful checking, we know that it's still possible for problems to occur or edge cases to be missed. We promise to fix any error that comes to our attention as quickly as practically possible. If there's an error that could lead to a potentially unsafe design, we'll contact you directly and tell you exactly which projects may be affected, so that you may take any action required as fast as possible.

Full Transparency

Formulas & References

For every field in a ClearCalcs calculator, clicking on the label will display the relevant formula, code or standard reference, and usually a description for the value:

quality_transparency_promise_debug

There are certain cases in which it is not practicable for us to show every formula we've used. In these cases, however, we will still show you as much data as we can relevant to the calculation. We are also committed to either making use of existing and tested open-source code, or to releasing all the code for our behind-the-scenes engineering computations as open-source. The open-source engineering code we currently make use of includes:

  • anaStruct: Used in all our beam and advanced column calculators, this is a finite element analysis (FEA) package that we use for determining the precise shear, moment, and axial demands, as well as deflections and reactions. It uses elastic beam elements, and all results are shown in plots on the relevant calculators.
  • sectionproperties: Used in our custom cross-section modules, this is an FEA package that we use for determining all the section properties for arbitrary sections that a user inputs. Based upon a shell mesh, which we display in plots on on the relevant calculators, it determines areas, moments of inertia / second moments of area, torsion properties, etc.
  • pyCUFSM: Used to calculate the buckling parameters in some of our cold-formed steel and aluminium calculators, this is our own code which we have released as open-source. It is a port of the open-source CUFSM software, originally written for Matlab.

Changelogs

Being a cloud-based platform allows us to update our platform on an ongoing basis, to provide you with new features or bug fixes sometimes within hours. Each time you open one of your projects, you will automatically be notified of any available updates to the calculators. Clicking on the changelog in the upgrade screen will show you all new features or bug fixes included in the update. Within each calculator, the full history of all updates we've made is also available by clicking "See Changes..." in the header.

quality_and_transparency_promise_changelog

Audit Logs

One of the benefits of our platform being entirely cloud-based is that it makes teamwork as seamless as possible. Multiple users can work on the same project from their own account. To support accountability within organisations, we have implemented audit logs, which show all the users who edited a design and when they did so.

[SUPPORT]_our_quality_and_transparency_promise_0.png

Ruthless Checking

Our most important task into providing a trustworthy platform is to make sure we provide trustworthy calculations from their initial release. We have developed a multi-step process applied universally throughout our platform to validate even the smallest change we make to our platform. For major changes, such as whenever a new calculator is released or when there are significant changes to a calculator, we'll additionally recruit an independent expert to check the calculation and we'll only publish the calculator with a "beta" tag for its first few weeks. Examples of significant changes would be adding glulam and composite lumber products to timber/wood calculators, or upgrading the cold-formed steel beam calculator upon a new major revision to the standard.

1. Automated Testing

For every update done on ClearCalcs, we run dozens of test cases on every single module - including historical versions - to ensure that no unexpected changes occur. This means more than a million tests running in total for every single minute change. With every new calculator, our team generates test cases to cover all foreseeable design conditions, usually based upon published design examples. For example, in our concrete wall footing module, we included test cases with varying load eccentricities, reinforced and plain concrete, different load types (e.g. snow, wind, earthquake), and hooked and epoxy coated rebar.

quality_and_transparency_promise_automated_testing

The slightest changes, down to the billionth of a percent, are flagged and must be validated by the relevant engineers on our team before the update can be released.

We also validate and automatically balance all units - avoiding any conversion errors, which are seen far too commonly in engineering. Expected units for every formula must be entered and if there is a mismatch, it will be detected and flagged before the module even begins testing.

2. Internal Peer Review

Once all tests are passed and accounted for, the calculator is passed to another engineer on our team for a peer review. This involves going over every calculation performed and comparing against the relevant standard, running through design examples and hand calculations, intentionally entering typos and unusual or impossible scenarios and seeing their effect, and thoroughly investigating any unexpected result. Their feedback will be passed back to the original author, who will then make appropriate corrections or clarifications, before requesting a re-review. This cycle may continue for multiple rounds in particularly complex changes or new calculators. Only once the peer reviewer is satisfied may the calculator be approved. For a minor change, the changes will then be published, but for any new calculators or major changes, we have two more checking

3. External Independent Review

After being created by our team and rigorously cross-checked against the relevant standards, literature, and hand computations, major changes will be handed over to one or more external and independent third party engineers with specific expertise in the area the calculator deals in. These third-party checkers thoroughly review the calculations, check output against their own computations that they would have signed off on, and provide feedback on design considerations and structure. Their feedback is passed to the original author, who will make appropriate corrections and clarifications, before requesting a re-review. Only once the third-party engineer(s) tell us that they would be fully comfortable signing off on a design built using our calculator do we accept this step as complete.

4. Beta Status

During the external independent review, we often release the calculator with a "Beta" tag and message, so that our users may preview what's upcoming, and have a chance to provide their own feedback on the calculation before the calculator is finalised. The Beta tag clearly denotes that these calculators have not yet completed our strenuous quality assurance process, and as such, we strongly advise against their use in production jobs. Once we are satisfied with the calculator, we will remove the beta tag and notify users of its availability. Designs created using the Beta calculator will retain the Beta tag until they are upgraded to the newest, fully-validated version.

quality_and_transparency_promise_beta_tag

ClearCalcs is a continuously improving platform in an ever-changing world of new engineering technologies. While we are constantly monitoring developments in relevant fields, nothing beats the combined experience of all our users in finding ways to improve our calculators, identifying needs to new calculators, or spotting problems and strange edge cases that aren't fully considered. We warmly welcome any feedback, and we prioritise our work largely based upon it!

Safety First

Even with the most careful checking, we know that it's still possible for problems to occur or edge cases to be missed. We prioritise these problems into three levels: minor bugs, bugs, and critical bugs. All bugs are addressed with the highest priority, before we touch any work on new features, and work to address them begins the moment we become aware of them.

Minor Bugs

A "minor bug" is anything that doesn't actually affect the results of a calculation - including typos, overlapping text in diagrams, an incorrect reference, or descriptions that aren't as clear as they should be. While correcting these still takes precedence over new features, we may take a few days to fix them. When we publish the correction to our platform, users will be notified in the sidebar when they open a project that an upgrade to their calculator is available, but it is entirely safe to ignore.

Bugs

A moderate bug is one which causes an error message, is extremely obvious to a lay observer, or which is always overly conservative. Examples include anything that causes a red error bar on the page, all demands on a member showing up as zero, or an overly-aggressive check that's being performed when it doesn't need to. Usually, these come to our attention in the form of a highly-unusual edge case that had not been properly considered. Fixing such problems is a top priority for us, and we aim to address them as fast as possible. If we expect a fix to require significant work, we may roll back a calculator to an earlier version while we work on a proper fix.

Critical Bugs

While our rigorous checking does make more severe bugs extremely rare, they are still possible. A "critical bug" is one in which there is a possibility of an incorrect result that is unconservative and not blatantly obvious being presented. Examples of situations considered critical might be data incorrectly copied from a manufacturer's catalog, or buckling checks somehow not being performed on a slender section. As this type of error could potentially affect the safety of designs, we take significant steps to address these as quickly and transparently as possible, and we commit to you that:

  • The moment we become aware of a critical bug, we will stop all current work and focus all our resources on fixing the bug.
  • The calculator will be immediately reverted to an earlier correct version and/or set as "Beta" to alert users that it should not be used for final designs at this time.
  • Every user with an offending calculation will be notified directly as soon as practically possible
    • To do this, we'll search our entire calculation database and narrow down the list of affected users based on the exact conditions causing the error
  • As soon as we release a fix to our platform, all users having an active copy of the problematic calculator, regardless of whether or not they encountered the bug, will be prompted to update to the corrected version.
  • After the bug is fully fixed, the relevant team will meet to review what happened and how it can be avoided in the future. Safeguards will be developed and implemented to ensure that the root cause of the issue is addressed.

quality_and_transparency_promise_critical_bugs_1

When upgrading the calculator, we will show the before and after comparison of utilisation in the member, so that any effects on the project are clearly visible:

quality_and_transparency_promise_critical_bugs_2

Related Articles

No items found.

Ready to ditch the spreadsheets?
So are we.

Experience the full power of ClearCalcs with a 14 day free trial and start being more productive.

Get Started for Free