Testing business-logic-in-dsls
-
Upload
mayank-jain -
Category
Software
-
view
112 -
download
0
Transcript of Testing business-logic-in-dsls
![Page 1: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/1.jpg)
Testing Business Logic Using DSLs in Clojure
Mayank Jain Test Engineer
![Page 2: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/2.jpg)
A TALK, IN 6 PARTS
1. What is the problem?
2. Real World Example.
3. Demo of the actual code.
4. Other features tested using same ideas.
5. Advantages/Disadvantages of writing DSLs for testing.
6. QA.
![Page 3: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/3.jpg)
WHAT IS THE PROBLEM?
Testing real world stateful business logic is hard
![Page 4: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/4.jpg)
Microwave Oven State Machine
Business logic is Stateful
![Page 5: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/5.jpg)
Microwave Oven State Machine
Large number transition states
![Page 6: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/6.jpg)
Difficult to enumerate all possible cases
Microwave Oven State Machine
![Page 7: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/7.jpg)
Tests become unreadable
Microwave Oven State Machine
![Page 8: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/8.jpg)
HELPSHIFT
• Embeddable support desk for native apps
• Main Features:
• Frequently Asked Questions which customers can search
• File issues/tickets from within the app
![Page 9: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/9.jpg)
Helpshift
Supercell
Clash of Clans
Gaana
General Billing
Domain
App
Section
FAQ 1 FAQ 2FAQs
Boom Beach Gaana App
Translations English Content Hindi Content
….
….
….
….
….
Top Down View of FAQs
![Page 10: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/10.jpg)
Example of Gaana FAQ Page
Customers can search FAQs
Available FAQs FAQ Sections
Domain
App
![Page 11: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/11.jpg)
FAQ Title
FAQ Body
FAQ is Visible?
![Page 12: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/12.jpg)
PROBLEM
You cannot share FAQs across apps.
Big Customers have multiple apps which have same FAQ translations content example “Privacy Policy”
![Page 13: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/13.jpg)
FEATURE: LINKED FAQS
![Page 14: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/14.jpg)
{:published? false :id “faq-id-1” :app_id “app-1” :section_id “section-1” :translations {:en {:published? false :stags [] :body “Privacy Body” :title “Privacy Title”}} :linked_faq_ids [“faq-id-2”] :publish_id “1”}
App-1
{:published? false :id “faq-id-2” :app_id “app-2” :section_id “section-2” :translations {:en {:published? false :stags [] :body “Privacy Body" :title “Privacy Title"}} :linked_faq_ids [“faq-id-1”] :publish_id “2”}
App-2
faq 1
(Sync faq-1 to App-2)General
faq 2
General
![Page 15: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/15.jpg)
{:published? false :id “faq-id-1” :app_id “app-1” :section_id “section-1” :translations {:en {:published? false :stags [] :body “Update Body” :title “Privacy Title”}} :linked_faq_ids [“faq-id-2”] :publish_id “1”}
{:published? false :id “faq-id-2” :app_id “app-2” :section_id “section-2” :translations {:en {:published? false :stags [] :body “Privacy Body" :title “Privacy Title"}} :linked_faq_ids [“faq-id-1”] :publish_id “2”}
faq 1Sync
faq 2
Update FAQ 1’s Body
![Page 16: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/16.jpg)
{:published? false :id “faq-id-1” :app_id “app-1” :section_id “section-1” :translations {:en {:published? false :stags [] :body “Update Body” :title “Privacy Title”}} :linked_faq_ids [“faq-id-2”] :publish_id “1”}
{:published? false :id “faq-id-2” :app_id “app-2” :section_id “section-2” :translations {:en {:published? false :stags [] :body “Update Body" :title “Privacy Title"}} :linked_faq_ids [“faq-id-1”] :publish_id “2”}
faq 1Sync
faq 2
Update FAQ 1’s Body
![Page 17: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/17.jpg)
EXAMPLE TEST CASE
![Page 18: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/18.jpg)
1. Add 1st App with only English languages.
2. Add 2nd App with only English languages
3. Add 1st FAQ under 1st App
4. Link 1st FAQ to 2nd App to create 2nd FAQ
5. Check -> translations of 2nd FAQ == 1st FAQ
6. Update English title of FAQ-1
7. Check -> translations of 2nd FAQ == 1st FAQ
8. Delete FAQ-1
9. Check if 1st FAQ is deleted in DB
10.Assert 2nd FAQ should remain as it is in database.
Simulation Verification
![Page 19: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/19.jpg)
PARTS OF EACH ACTION
“Add APP-1 with only English language”
1. Type of Action Add an App
2. Names App-1
3. Arguments English Language
4. Expected Result As per Spec/Modal of the system
5. Actual Result Database
![Page 20: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/20.jpg)
EXPRESS ACTIONS AS CLOJURE DATA
“Add app APP-1 with English language”:add-app :app-1 {:langs-config [:en]}
![Page 21: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/21.jpg)
ONE UNIT OF ACTION
[:add-app :app-1 {:langs-config [:en]}]
![Page 22: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/22.jpg)
SERIES OF ACTIONS[[:add-app :app-1 {:langs-config [:en]]
[:add-app :app-2 {:langs-config nil}]
[:add-faq :faq-1 {:app-var :app-1 ...}]
[:link-faq :faq-2 {:faq-var :faq-1 :app-var :app-2 ...}]
[:update-faq nil {:app-var :app-1 :faq var :faq-1 ...}]
[:update-faq nil {:app-var :app-2 :faq-var :faq-2 ...}]
[:delete-faq nil {:faq-var :faq-1 ...}]]
![Page 23: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/23.jpg)
REDUCE ON ACTIONS
Reduce
{}
[:add-app :app-1 {:langs-config nil}])
{:env {:apps {:app-1 {…}}} :result [{result-1…}]} …..}
Reduce
[:add-app :app-2 {:langs-config nil}])
{:env {:apps {:app-1 {…} :app-2 {…}}} :result [{result-1…} {result-2…}]} …..}
{:env {:apps {:app-1 {…}}} :result [{result-1…}]} …..}
And so on…
![Page 24: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/24.jpg)
RESULT HASH-MAP COMPRISES OF
{:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}}} :faqs {:faq-1 {:faq-id “112”}}}} :result [{:action-type :add-faq :expected {….} :actual {….}} {…more}] “112” {…state…} …more}
Environment (:env) - Contains bindings of vars
![Page 25: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/25.jpg)
RESULT HASH-MAP COMPRISES OF
{:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}}} :faqs {:faq-1 {:faq-id “112”}}}} :result [{:action-type :add-faq :expected {….} :actual {….}} {…more}] “112” {…state…} …more}
Result (:result) - Contains Actual And Expected Result
![Page 26: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/26.jpg)
RESULT HASH-MAP COMPRISES OF
{:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}}} :faqs {:faq-1 {:faq-id “112”}}}} :result [{:action-type :add-faq :expected {….} :actual {….}} {…more}] “112” {…state…} …more}
Current Generated Expected State
![Page 27: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/27.jpg)
Dispatch On Action Type
Update The Variables in arguments to its
bindings
Call args with relevant function
Bind the result to the given var in Global Data
Store Expected current state
Store Actual current database state
What happens
inside the
reducer?
![Page 28: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/28.jpg)
Dispatch On Action Type
Update The Variables in arguments to its
bindings
Call args with relevant function
Bind the result to the given var in Global Data
Store Expected current state
Store Actual current database state
[:add-app :app-1 {:langs-config nil}])
[:add-app :app-1 {:langs-config nil}])
(add-app {:langs-config nil})
{:env {:apps {:app-1 {:app-id "970"}}}}
![Page 29: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/29.jpg)
Dispatch On Action Type
Update The Variables in arguments to its
bindings
Call args with relevant function
Bind the result to the given var in Global Data
Store Expected current state
Store Actual current database state
[:add-app :app-2 {:langs-config nil}])
[:add-app :app-2 {:langs-config nil}])
(add-app {:langs-config nil})
{:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142"}}}}
![Page 30: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/30.jpg)
Dispatch On Action Type
Update The Variables in arguments to its
bindings
[:add-faq :faq-1 {:app-var :app-1 ….}])
{:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}}}
![Page 31: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/31.jpg)
Dispatch On Action Type
Update The Variables in arguments to its
bindings
[:add-faq :faq-1 {:app-var :app-1 ….}])
[:add-faq :faq-1 {:app-id “970” …}])
{:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}}}
![Page 32: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/32.jpg)
Dispatch On Action Type
Update The Variables in arguments to its
bindings
Call args with relevant function
Bind the result to the given var in Global Data
[:add-faq :faq-1 {:app-var :app-1 ….}])
[:add-faq :faq-1 {:app-id “970” …}])
(add-faq {:app-id “970” ….})
![Page 33: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/33.jpg)
Dispatch On Action Type
Update The Variables in arguments to its
bindings
Call args with relevant function
Bind the result to the given var in Global Data
[:add-faq :faq-1 {:app-var :app-1 ….}])
[:add-faq :faq-1 {:app-id “970” …}])
(add-faq {:app-id “970” ….}){:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}} :faqs {:faq-1 {:faq-id “112”}}}}
![Page 34: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/34.jpg)
Dispatch On Action Type
Update The Variables in arguments to its
bindings
Call args with relevant function
Bind the result to the given var in Global Data
Store Expected current state
[:add-faq :faq-1 {:app-var :app-1 ….}])
[:add-faq :faq-1 {:app-id “970” …}])
(add-faq {:app-id “970” ….}){:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}} :faqs {:faq-1 {:faq-id “112”}}}}{:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}}} :faqs {:faq-1 {:faq-id “112”}}}} :result [{:action-type :add-faq :expected {….}}
![Page 35: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/35.jpg)
Dispatch On Action Type
Update The Variables in arguments to its
bindings
Call args with relevant function
Bind the result to the given var in Global Data
Store Expected current state
Store Actual current database state
[:add-faq :faq-1 {:app-var :app-1 ….}])
[:add-faq :faq-1 {:app-id “970” …}])
(add-faq {:app-id “970” ….}){:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}} :faqs {:faq-1 {:faq-id “112”}}}}{:env {:apps {:app-1 {:app-id “970"} :app-2 {:app-id “142”}}} :faqs {:faq-1 {:faq-id “112”}}}} :result [{:action-type :add-faq :expected {….} :actual {….}}
![Page 36: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/36.jpg)
VERIFICATION:COMPARE EXPECTED VS ACTUAL FOR
EACH STEP
![Page 37: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/37.jpg)
COMPARE RESULT FORFIRST ACTION
{"112" {:translations {:en {:published? false, :stags [], :body "Temp body", :title “Title 1"}}, :linked-faq-ids #{}}}
Actual Data in DBExpected Data
from my Generated result
{"112" {:translations {:en {:published? false, :stags [], :body "Temp body", :title “Title 1"}}, :linked-faq-ids #{}}}
==PASS
![Page 38: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/38.jpg)
{"112" {:translations {:en {:published? false, :stags [], :body "Temp body", :title "Title 1"}}, :linked-faq-ids #{}}}
Actual Data in DBExpected Data
from my Generated result
{"112" {:translations {:en {:published? false, :stags [], :body "Temp body", :title "Title 2"}}, :linked-faq-ids #{}}}
==FAIL
COMPARE RESULT FORSECOND ACTION
![Page 39: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/39.jpg)
DEMO TIME
![Page 40: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/40.jpg)
APPROACHES TO VERIFY
Actions : HardcodedExpected Output : Hardcoded
Actions : SimulatedExpected Output : Hardcoded
Actions : SimulatedExpected Output : Generated
Actions : GeneratedExpected Output : Generated
![Page 41: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/41.jpg)
Hardcoded Actions, Hardcoded Expected Output
ADVANTAGES DISADVANTAGES
Requires no extra knowledge to understand
Very cumbersome to enumerate
Anyone can add/edit tests Modification is hard
False negatives are not possible
![Page 42: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/42.jpg)
Simulate list of Actions, Hardcoded Final Expected
Output
ADVANTAGES DISADVANTAGES
Easy to enumerateRequires knowledge of
the DSL
Very readableMaintenance overhead of
DSL
Shareable with dev to simulate bugs
Does not check intermediate state
False negatives are not possible
Expected output may have data which is
available only at runtime like faq-ids
![Page 43: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/43.jpg)
Simulate list of actions, Generate Expected Output
ADVANTAGES DISADVANTAGES
Easy to enumerateRequires knowledge of
the DSL
Very readableMaintenance overhead of
DSL
Shareable with dev to simulate bugs
Maintenance overhead of Expected Modal
Checks intermediate stateFalse negatives are
possibleRuntime data is available
like faq-ids
![Page 44: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/44.jpg)
Generate Actions, Generate Expected Output
ADVANTAGES DISADVANTAGES
Possible to generate large number of tests
Requires knowledge of the DSL
Possible to Shrink failed test case using test.check library
Maintenance overhead of DSL
Shareable with dev to simulate bugs
Maintenance overhead of Expected Modal
Checks intermediate stateFalse negatives are
possible
Maintenance overhead of generative code for list of
actions.
![Page 45: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/45.jpg)
Generate Actions, Generate Expected Output
Feature - Issue Audit Trail
![Page 46: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/46.jpg)
Issue Audit Trail• Maintains a log of
• Who Took an action
• What Action they took
![Page 47: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/47.jpg)
Who - Types of Users
App User Support User
Agents
Admins
Helpshift
![Page 48: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/48.jpg)
What - Types of Actions
App User
• Create Issue
• Reply Issue
• etc…
Admin User
• Create Issue
• Reply Issue
• Resolve Issue
• Edit Tags
• etc…
Agent User
• Create Issue
• Reply Issue
• Resolve Issue
• Edit Tags
• etc…
![Page 49: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/49.jpg)
App User
Admin“Mayank
”
Agent“Agent-2”
Example Issue
![Page 50: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/50.jpg)
Example Issue
Issue Audit Logs
![Page 51: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/51.jpg)
DEMO - ISSUE AUDIT TRAIL
Generate Actions, Generate Expected Output
![Page 52: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/52.jpg)
ADVANTAGES OF WRITING DSL
Discovering very hard to find bugs, for example we found:
Duplicate issue messages being rendered only if the number of messages were "just right".
Finding ordering bugs.
Increase in developer/tester productivity
![Page 53: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/53.jpg)
DISADVANTAGES OF WRITING DSL
Cost of maintaining DSLs is high.
If your DSL is just data being evaluated at run time, changes in feature code will not throw any compile time errors like function parameters being changed in DSL code.
You have to educate your team members to learn your specific DSL for that feature to be able to understand the tests.
![Page 54: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/54.jpg)
CONCLUSION
DSL can be used as a workflow for writing tests.
Separating simulation vs verification of tests.
A step towards ability to generate tests instead of writing them.
![Page 55: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/55.jpg)
Further Resources• Clojure Made Simple - Rich Hickey
• Growing a DSL with Clojure
• Jeanine Adkisson - Design and Prototype a Language In Clojure
• John Hughes - Testing the Hard Stuff and Staying Sane
• Reid Draper - Powerful Testing with test.check
• Clojure Tutorials on DSLs with Tim Baldridge (Paid)
![Page 56: Testing business-logic-in-dsls](https://reader038.fdocuments.net/reader038/viewer/2022110203/55d1fef7bb61eb48718b47a1/html5/thumbnails/56.jpg)
Any Questions?@firesofmay
facebook.com/firesofmay/