tag:blogger.com,1999:blog-58378652956762402652024-03-14T05:18:56.945+11:00Preposterous! Egregious!Consecutive sentences or phrases, usually pertaining to a topic or subject.David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.comBlogger22125tag:blogger.com,1999:blog-5837865295676240265.post-60689184427068309792018-07-26T09:39:00.000+10:002018-07-26T10:46:58.144+10:00Enforcing rules at compile-time: an example<style>
.goost{font-size:16px;color:#333;text-align:left;/*!
* GitHub Light v0.4.1
* Copyright (c) 2012 - 2017 GitHub, Inc.
* Licensed under MIT (https://github.com/primer/github-syntax-theme-generator/blob/master/LICENSE)
*/direction:ltr}.goost .markdown-body{font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol";font-size:16px;line-height:1.5;word-wrap:break-word}.goost .markdown-body::before{display:table;content:""}.goost .markdown-body::after{display:table;clear:both;content:""}.goost .markdown-body>*:first-child{margin-top:0 !important}.goost .markdown-body>*:last-child{margin-bottom:0 !important}.goost .markdown-body a:not([href]){color:inherit;text-decoration:none}.goost .markdown-body .absent{color:#cb2431}.goost .markdown-body .anchor{float:left;padding-right:4px;margin-left:-20px;line-height:1}.goost .markdown-body .anchor:focus{outline:none}.goost .markdown-body p,.goost .markdown-body blockquote,.goost .markdown-body ul,.goost .markdown-body ol,.goost .markdown-body dl,.goost .markdown-body table,.goost .markdown-body pre{margin-top:0;margin-bottom:16px}.goost .markdown-body hr{height:.25em;padding:0;margin:24px 0;background-color:#e1e4e8;border:0}.goost .markdown-body blockquote{padding:0 1em;color:#6a737d;border-left:0.25em solid #dfe2e5}.goost .markdown-body blockquote>:first-child{margin-top:0}.goost .markdown-body blockquote>:last-child{margin-bottom:0}.goost .markdown-body kbd{display:inline-block;padding:3px 5px;font-size:11px;line-height:10px;color:#444d56;vertical-align:middle;background-color:#fafbfc;border:solid 1px #c6cbd1;border-bottom-color:#959da5;border-radius:3px;box-shadow:inset 0 -1px 0 #959da5}.goost .markdown-body h1,.goost .markdown-body h2,.goost .markdown-body h3,.goost .markdown-body h4,.goost .markdown-body h5,.goost .markdown-body h6{margin-top:24px;margin-bottom:16px;font-weight:600;line-height:1.25}.goost .markdown-body h1 .octicon-link,.goost .markdown-body h2 .octicon-link,.goost .markdown-body h3 .octicon-link,.goost .markdown-body h4 .octicon-link,.goost .markdown-body h5 .octicon-link,.goost .markdown-body h6 .octicon-link{color:#1b1f23;vertical-align:middle;visibility:hidden}.goost .markdown-body h1:hover .anchor,.goost .markdown-body h2:hover .anchor,.goost .markdown-body h3:hover .anchor,.goost .markdown-body h4:hover .anchor,.goost .markdown-body h5:hover .anchor,.goost .markdown-body h6:hover .anchor{text-decoration:none}.goost .markdown-body h1:hover .anchor .octicon-link,.goost .markdown-body h2:hover .anchor .octicon-link,.goost .markdown-body h3:hover .anchor .octicon-link,.goost .markdown-body h4:hover .anchor .octicon-link,.goost .markdown-body h5:hover .anchor .octicon-link,.goost .markdown-body h6:hover .anchor .octicon-link{visibility:visible}.goost .markdown-body h1 tt,.goost .markdown-body h1 code,.goost .markdown-body h2 tt,.goost .markdown-body h2 code,.goost .markdown-body h3 tt,.goost .markdown-body h3 code,.goost .markdown-body h4 tt,.goost .markdown-body h4 code,.goost .markdown-body h5 tt,.goost .markdown-body h5 code,.goost .markdown-body h6 tt,.goost .markdown-body h6 code{font-size:inherit}.goost .markdown-body h1{padding-bottom:0.3em;font-size:2em;border-bottom:1px solid #eaecef}.goost .markdown-body h2{padding-bottom:0.3em;font-size:1.5em;border-bottom:1px solid #eaecef}.goost .markdown-body h3{font-size:1.25em}.goost .markdown-body h4{font-size:1em}.goost .markdown-body h5{font-size:0.875em}.goost .markdown-body h6{font-size:0.85em;color:#6a737d}.goost .markdown-body ul,.goost .markdown-body ol{padding-left:2em}.goost .markdown-body ul.no-list,.goost .markdown-body ol.no-list{padding:0;list-style-type:none}.goost .markdown-body ul ul,.goost .markdown-body ul ol,.goost .markdown-body ol ol,.goost .markdown-body ol ul{margin-top:0;margin-bottom:0}.goost .markdown-body li{word-wrap:break-all}.goost .markdown-body li>p{margin-top:16px}.goost .markdown-body li+li{margin-top:.25em}.goost .markdown-body dl{padding:0}.goost .markdown-body dl dt{padding:0;margin-top:16px;font-size:1em;font-style:italic;font-weight:600}.goost .markdown-body dl dd{padding:0 16px;margin-bottom:16px}.goost .markdown-body table{display:block;width:100%;overflow:auto}.goost .markdown-body table th{font-weight:600}.goost .markdown-body table th,.goost .markdown-body table td{padding:6px 13px;border:1px solid #dfe2e5}.goost .markdown-body table tr{background-color:#fff;border-top:1px solid #c6cbd1}.goost .markdown-body table tr:nth-child(2n){background-color:#f6f8fa}.goost .markdown-body table img{background-color:transparent}.goost .markdown-body img{max-width:100%;box-sizing:content-box;background-color:#fff}.goost .markdown-body img[align=right]{padding-left:20px}.goost .markdown-body img[align=left]{padding-right:20px}.goost .markdown-body .emoji{max-width:none;vertical-align:text-top;background-color:transparent}.goost .markdown-body span.frame{display:block;overflow:hidden}.goost .markdown-body span.frame>span{display:block;float:left;width:auto;padding:7px;margin:13px 0 0;overflow:hidden;border:1px solid #dfe2e5}.goost .markdown-body span.frame span img{display:block;float:left}.goost .markdown-body span.frame span span{display:block;padding:5px 0 0;clear:both;color:#24292e}.goost .markdown-body span.align-center{display:block;overflow:hidden;clear:both}.goost .markdown-body span.align-center>span{display:block;margin:13px auto 0;overflow:hidden;text-align:center}.goost .markdown-body span.align-center span img{margin:0 auto;text-align:center}.goost .markdown-body span.align-right{display:block;overflow:hidden;clear:both}.goost .markdown-body span.align-right>span{display:block;margin:13px 0 0;overflow:hidden;text-align:right}.goost .markdown-body span.align-right span img{margin:0;text-align:right}.goost .markdown-body span.float-left{display:block;float:left;margin-right:13px;overflow:hidden}.goost .markdown-body span.float-left span{margin:13px 0 0}.goost .markdown-body span.float-right{display:block;float:right;margin-left:13px;overflow:hidden}.goost .markdown-body span.float-right>span{display:block;margin:13px auto 0;overflow:hidden;text-align:right}.goost .markdown-body code,.goost .markdown-body tt{padding:0.2em 0.4em;margin:0;font-size:85%;background-color:rgba(27,31,35,0.05);border-radius:3px}.goost .markdown-body code br,.goost .markdown-body tt br{display:none}.goost .markdown-body del code{text-decoration:inherit}.goost .markdown-body pre{word-wrap:normal}.goost .markdown-body pre>code{padding:0;margin:0;font-size:100%;word-break:normal;white-space:pre;background:transparent;border:0}.goost .markdown-body .highlight{margin-bottom:16px}.goost .markdown-body .highlight pre{margin-bottom:0;word-break:normal}.goost .markdown-body .highlight pre,.goost .markdown-body pre{padding:16px;overflow:auto;font-size:85%;line-height:1.45;background-color:#f6f8fa;border-radius:3px}.goost .markdown-body pre code,.goost .markdown-body pre tt{display:inline;max-width:auto;padding:0;margin:0;overflow:visible;line-height:inherit;word-wrap:normal;background-color:transparent;border:0}.goost .markdown-body .csv-data td,.goost .markdown-body .csv-data th{padding:5px;overflow:hidden;font-size:12px;line-height:1;text-align:left;white-space:nowrap}.goost .markdown-body .csv-data .blob-num{padding:10px 8px 9px;text-align:right;background:#fff;border:0}.goost .markdown-body .csv-data tr{border-top:0}.goost .markdown-body .csv-data th{font-weight:600;background:#f6f8fa;border-top:0}.goost .pl-c{color:#6a737d}.goost .pl-c1,.goost .pl-s .pl-v{color:#005cc5}.goost .pl-e,.goost .pl-en{color:#6f42c1}.goost .pl-smi,.goost .pl-s .pl-s1{color:#24292e}.goost .pl-ent{color:#22863a}.goost .pl-k{color:#d73a49}.goost .pl-s,.goost .pl-pds,.goost .pl-s .pl-pse .pl-s1,.goost .pl-sr,.goost .pl-sr .pl-cce,.goost .pl-sr .pl-sre,.goost .pl-sr .pl-sra{color:#032f62}.goost .pl-v,.goost .pl-smw{color:#e36209}.goost .pl-bu{color:#b31d28}.goost .pl-ii{color:#fafbfc;background-color:#b31d28}.goost .pl-c2{color:#fafbfc;background-color:#d73a49}.goost .pl-c2::before{content:"^M"}.goost .pl-sr .pl-cce{font-weight:bold;color:#22863a}.goost .pl-ml{color:#735c0f}.goost .pl-mh,.goost .pl-mh .pl-en,.goost .pl-ms{font-weight:bold;color:#005cc5}.goost .pl-mi{font-style:italic;color:#24292e}.goost .pl-mb{font-weight:bold;color:#24292e}.goost .pl-md{color:#b31d28;background-color:#ffeef0}.goost .pl-mi1{color:#22863a;background-color:#f0fff4}.goost .pl-mc{color:#e36209;background-color:#ffebda}.goost .pl-mi2{color:#f6f8fa;background-color:#005cc5}.goost .pl-mdr{font-weight:bold;color:#6f42c1}.goost .pl-ba{color:#586069}.goost .pl-sg{color:#959da5}.goost .pl-corl{text-decoration:underline;color:#032f62}.goost .breadcrumb{margin-bottom:10px;font-size:18px;color:#586069}.goost .breadcrumb .separator::before,.goost .breadcrumb .separator::after{content:" "}.goost .breadcrumb strong.final-path{color:#24292e}.goost .breadcrumb .zeroclipboard-button{display:inline-block;margin-left:5px}.goost .breadcrumb .repo-root{font-weight:600}.goost .breadcrumb .octicon{vertical-align:-2px}.goost .editor-license-template,.goost .editor-code-of-conduct-template,.goost .editor-gitignore-template{position:relative;top:3px;display:block;float:right;font-size:14px}.goost .editor-license-template .select-menu-git-ignore,.goost .editor-code-of-conduct-template .select-menu-git-ignore,.goost .editor-gitignore-template .select-menu-git-ignore{right:0}.goost .editor-abort{display:inline;font-size:14px}.goost .blob-interaction-bar{position:relative;background-color:#f2f2f2;border-bottom:1px solid #e5e5e5}.goost .blob-interaction-bar::before{display:table;content:""}.goost .blob-interaction-bar::after{display:table;clear:both;content:""}.goost .blob-interaction-bar .octicon-search{position:absolute;top:10px;left:10px;font-size:12px;color:#586069}.goost .blob-filter{width:100%;padding:4px 20px 5px 30px;font-size:12px;border:0;border-radius:0;outline:none}.goost .blob-filter:focus{outline:none}.goost .html-blob{margin-bottom:15px}.goost .TagsearchPopover-content{width:inherit;max-width:600px}.goost .license-summary-octicon{color:#959da5}.goost .rule-type-permissions{color:#28a745}.goost .rule-type-conditions{color:#0366d6}.goost .rule-type-limitations{color:#d73a49}.goost .blob-wrapper{overflow-x:auto;overflow-y:hidden;border-bottom-right-radius:3px;border-bottom-left-radius:3px}.goost .blob-wrapper-embedded{max-height:240px;overflow-y:auto}.goost .diff-table{width:100%;border-collapse:separate}.goost .diff-table .line-comments{padding:10px;vertical-align:top;border-top:1px solid #e1e4e8}.goost .diff-table .line-comments:first-child+.empty-cell{border-left-width:1px}.goost .diff-table tr:not(:last-child) .line-comments{border-top:1px solid #e1e4e8;border-bottom:1px solid #e1e4e8}.goost .blob-num{width:1%;min-width:50px;padding-right:10px;padding-left:10px;font-family:"SFMono-Regular",Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:12px;line-height:20px;color:rgba(27,31,35,0.3);text-align:right;white-space:nowrap;vertical-align:top;cursor:pointer;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}.goost .blob-num:hover{color:rgba(27,31,35,0.6)}.goost .blob-num::before{content:attr(data-line-number)}.goost .blob-num.non-expandable{cursor:default}.goost .blob-num.non-expandable:hover{color:rgba(27,31,35,0.3)}.goost .blob-code{position:relative;padding-right:10px;padding-left:10px;line-height:20px;vertical-align:top}.goost .blob-code-inner{overflow:visible;font-family:"SFMono-Regular",Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:12px;color:#24292e;word-wrap:normal;white-space:pre}.goost .blob-code-inner .x-first{border-top-left-radius:0.2em;border-bottom-left-radius:0.2em}.goost .blob-code-inner .x-last{border-top-right-radius:0.2em;border-bottom-right-radius:0.2em}.goost .blob-code-inner::before{content:""}.goost .blob-code-inner.highlighted{background-color:#fffbdd}.goost .blob-code-marker-addition::before{content:"+ "}.goost .blob-code-marker-deletion::before{content:"- "}.goost .blob-code-marker-context::before{content:" "}.goost .soft-wrap .diff-table{table-layout:fixed}.goost .soft-wrap .blob-code{padding-left:18px;text-indent:-7px}.goost .soft-wrap .blob-code-inner{word-wrap:break-word;white-space:pre-wrap}.goost .soft-wrap .no-nl-marker{display:none}.goost .soft-wrap .add-line-comment{margin-left:-28px}.goost .blob-num-hunk,.goost .blob-code-hunk,.goost .blob-num-expandable,.goost .blob-code-expandable{color:rgba(27,31,35,0.5);vertical-align:middle}.goost .blob-num-hunk,.goost .blob-num-expandable{background-color:#dbedff}.goost .blob-code-hunk,.goost .blob-code-expandable{padding-top:4px;padding-bottom:4px;background-color:#f1f8ff;border-width:1px 0}.goost .blob-expanded .blob-num,.goost .blob-expanded .blob-code{background-color:#fafbfc}.goost .blob-expanded+tr:not(.blob-expanded) .blob-num,.goost .blob-expanded+tr:not(.blob-expanded) .blob-code{border-top:1px solid #eaecef}.goost .blob-expanded .blob-num-hunk{border-top:1px solid #eaecef}.goost tr:not(.blob-expanded)+.blob-expanded .blob-num,.goost tr:not(.blob-expanded)+.blob-expanded .blob-code{border-top:1px solid #eaecef}.goost .blob-num-expandable{padding:0;font-size:12px;text-align:center}.goost .blob-num-expandable .octicon{vertical-align:top}.goost .blob-num-expandable .diff-expander{display:block;width:auto;height:auto;padding:4px 11px 4px 10px;margin-right:-1px;color:#586069;cursor:pointer}.goost .blob-num-expandable .diff-expander:hover{color:#fff;text-shadow:none;background-color:#0366d6;border-color:#0366d6}.goost .blob-code-addition{background-color:#e6ffed}.goost .blob-code-addition .x{color:#24292e;background-color:#acf2bd}.goost .blob-num-addition{background-color:#cdffd8;border-color:#bef5cb}.goost .blob-code-deletion{background-color:#ffeef0}.goost .blob-code-deletion .x{color:#24292e;background-color:#fdb8c0}.goost .blob-num-deletion{background-color:#ffdce0;border-color:#fdaeb7}.goost .selected-line.blob-code{background-color:#fffbdd}.goost .selected-line.blob-code .x{background-color:transparent}.goost .selected-line.blob-num{background-color:#fff5b1;border-color:#ffea7f}.goost .add-line-comment{position:relative;z-index:5;float:left;width:22px;height:22px;margin:-2px -10px -2px -20px;line-height:21px;color:#fff;text-align:center;text-indent:0;cursor:pointer;background-color:#0366d6;background-image:linear-gradient(#0372ef, #0366d6);border-radius:3px;box-shadow:0 1px 4px rgba(27,31,35,0.15);opacity:0;transition:transform 0.1s ease-in-out;transform:scale(0.8, 0.8)}.goost .add-line-comment:hover{transform:scale(1, 1)}.is-hovered .goost .add-line-comment,.goost .add-line-comment:focus{opacity:1}.goost .add-line-comment .octicon{vertical-align:text-top;pointer-events:none}.goost .add-line-comment.octicon-check{background:#333;opacity:1}.goost .inline-comment-form{border:1px solid #dfe2e5;border-radius:3px}.goost .inline-review-comment{margin-top:0 !important;margin-bottom:10px !important}.goost .inline-review-comment .gc:first-child+tr .blob-num,.goost .inline-review-comment .gc:first-child+tr .blob-code{padding-top:5px}.goost .inline-review-comment tr:last-child{border-bottom-right-radius:2px;border-bottom-left-radius:2px}.goost .inline-review-comment tr:last-child .blob-num,.goost .inline-review-comment tr:last-child .blob-code{padding-bottom:8px}.goost .inline-review-comment tr:last-child .blob-num:first-child,.goost .inline-review-comment tr:last-child .blob-code:first-child{border-bottom-left-radius:2px}.goost .inline-review-comment tr:last-child .blob-num:last-child,.goost .inline-review-comment tr:last-child .blob-code:last-child{border-bottom-right-radius:2px}.goost .timeline-inline-comments{width:100%;table-layout:fixed}.goost .timeline-inline-comments .inline-comments,.goost .show-inline-notes .inline-comments{display:table-row}.goost .inline-comments{display:none}.goost .inline-comments.is-collapsed{display:none}.goost .inline-comments .line-comments.is-collapsed{visibility:hidden}.goost .inline-comments .line-comments+.blob-num{border-left-width:1px}.goost .inline-comments .timeline-comment{margin-bottom:10px}.goost .inline-comments .inline-comment-form,.goost .inline-comments .inline-comment-form-container{max-width:780px}.goost .comment-holder{max-width:780px}.goost .line-comments+.line-comments,.goost .empty-cell+.line-comments{border-left:1px solid #eaecef}.goost .inline-comment-form-container .inline-comment-form,.goost .inline-comment-form-container.open .inline-comment-form-actions{display:none}.goost .inline-comment-form-container .inline-comment-form-actions,.goost .inline-comment-form-container.open .inline-comment-form{display:block}.goost body.split-diff .container,.goost body.split-diff .container-lg,.goost body.full-width .container,.goost body.full-width .container-lg{width:100%;max-width:none;padding-right:20px;padding-left:20px}.goost body.split-diff .repository-content,.goost body.full-width .repository-content{width:100%}.goost body.split-diff .new-pr-form,.goost body.full-width .new-pr-form{max-width:980px}.goost body.split-diff .new-pr-form .discussion-sidebar,.goost body.full-width .new-pr-form .discussion-sidebar{width:200px}.goost .file-diff-split{table-layout:fixed}.goost .file-diff-split .blob-code+.blob-num{border-left:1px solid #f6f8fa}.goost .file-diff-split .blob-code-inner{word-wrap:break-word;white-space:pre-wrap}.goost .file-diff-split .empty-cell{cursor:default;background-color:#fafbfc;border-right-color:#eaecef}.goost .submodule-diff-stats .octicon-diff-removed{color:#cb2431}.goost .submodule-diff-stats .octicon-diff-renamed{color:#677a85}.goost .submodule-diff-stats .octicon-diff-modified{color:#d0b44c}.goost .submodule-diff-stats .octicon-diff-added{color:#28a745}.goost .BlobToolbar{left:-17px}.goost .BlobToolbar-dropdown{margin-left:-2px}.goost .task-list-item{list-style-type:none}.goost .task-list-item label{font-weight:400}.goost .task-list-item.enabled label{cursor:pointer}.goost .task-list-item+.task-list-item{margin-top:3px}.goost .task-list-item .handle{display:none}.goost .task-list-item-checkbox{margin:0 0.2em 0.25em -1.6em;vertical-align:middle}.goost .reorderable-task-lists .markdown-body .contains-task-list{padding:0}.goost .reorderable-task-lists .markdown-body li:not(.task-list-item){margin-left:26px}.goost .reorderable-task-lists .markdown-body ol:not(.contains-task-list) li,.goost .reorderable-task-lists .markdown-body ul:not(.contains-task-list) li{margin-left:0}.goost .reorderable-task-lists .markdown-body li p{margin-top:0}.goost .reorderable-task-lists .markdown-body .task-list-item{padding-right:15px;padding-left:42px;margin-right:-15px;margin-left:-15px;border:1px solid transparent}.goost .reorderable-task-lists .markdown-body .task-list-item+.task-list-item{margin-top:0}.goost .reorderable-task-lists .markdown-body .task-list-item .contains-task-list{padding-top:4px}.goost .reorderable-task-lists .markdown-body .task-list-item .handle{display:block;float:left;width:20px;padding:2px 0 0 2px;margin-left:-43px;opacity:0}.goost .reorderable-task-lists .markdown-body .task-list-item .drag-handle{fill:#333}.goost .reorderable-task-lists .markdown-body .task-list-item.hovered{background:#fafafa;border-top-color:#ededed;border-bottom-color:#ededed}.goost .reorderable-task-lists .markdown-body .task-list-item.hovered>.handle{opacity:1}.goost .reorderable-task-lists .markdown-body .task-list-item.is-dragging{opacity:0}.goost .reorderable-task-lists .markdown-body .task-list-item.is-ghost{border-right-color:#ededed;border-left-color:#ededed}.goost .review-comment-contents .markdown-body .task-list-item{padding-left:42px;margin-right:-12px;margin-left:-12px;border-top-left-radius:3px;border-bottom-left-radius:3px}.goost .review-comment-contents .markdown-body .task-list-item.hovered{border-left-color:#ededed}.goost .highlight{padding:0;margin:0;font-family:"SFMono-Regular",Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:12px;font-weight:400;line-height:1.4;color:#333;background:#fff;border:0}.goost .render-viewer-error,.goost .render-viewer-fatal,.goost .render-viewer-invalid,.goost .octospinner{display:none}.goost iframe.render-viewer{width:100%;height:480px;overflow:hidden;border:0}.goost pre,.goost code{font-family:"SFMono-Regular",Consolas,"Liberation Mono",Menlo,Courier,monospace !important;white-space:pre}.goost .goost-meta{padding:10px;overflow:hidden;font:12px -apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol";color:#586069;background-color:#f7f7f7;border-radius:0 0 2px 2px}.goost .goost-meta a{font-weight:600;color:#666;text-decoration:none;border:0}.goost .goost-data{overflow:auto;word-wrap:normal;background-color:#fff;border-bottom:1px solid #ddd;border-radius:2px 2px 0 0}.goost .goost-file{margin-bottom:1em;font-family:"SFMono-Regular",Consolas,"Liberation Mono",Menlo,Courier,monospace;border:1px solid #ddd;border-bottom:1px solid #ccc;border-radius:3px}.goost .goost-file article{padding:6px}.goost .goost-file .scroll .goost-data{position:absolute;top:0;right:0;bottom:30px;left:0;overflow:scroll}.goost .goost-file .scroll .goost-meta{position:absolute;right:0;bottom:0;left:0}.goost .blob-num{min-width:inherit;padding:1px 10px !important;background:transparent}.goost .blob-code{padding:1px 10px !important;text-align:left;background:transparent;border:0}.goost .blob-wrapper table{border-collapse:collapse}.goost .blob-wrapper tr:first-child td{padding-top:4px}.goost .markdown-body .anchor{display:none}
/*# sourceMappingURL=gist-embed-f66ab13f994260dfbd79394e5363ab20.css.map */
.goost .goost-file {border:none}
.goost .goost-data {border-bottom:none}
.goost .markdown-body .highlight pre, .goost .markdown-body pre {font-size:0.85rem}
</style>
<div id="goost84186183" class="goost">
<div class="goost-file">
<div class="goost-data">
<div class="js-goost-file-update-container js-task-list-container file-box">
<div id="file-blog-md" class="file">
<div id="readme" class="readme blob instapaper_body">
<article class="markdown-body entry-content" itemprop="text"><p>I recently solved a problem I had in Scala.
I was able to solve the problem quickly and easily, where as I remember when I was less experienced with Scala,
solving a problem like this was difficult. It would take a lot of thought and
effort, and even if I came away with a solution that technically worked,
it often felt off and wasn't pleasant to use.</p>
<p>I thought it'd be nice to share this example of my current approach to writing good Scala.
I hope you find it useful.</p>
<h1><a id="user-content-premise" class="anchor" aria-hidden="true" href="#premise"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Premise</h1>
<p>I'm the author and maintainer of <a href="https://github.com/japgolly/scalajs-react">scalajs-react</a> which is a Scala.JS library
that provides a type-safe interface to <a href="https://reactjs.org" rel="nofollow">React</a>, a JavaScript UI library.</p>
<p>In <a href="https://github.com/japgolly/scalajs-react">scalajs-react</a>, the primary way to create a UI component is via "the builder
pattern". The builder API is separated into 4 steps so that you first specify
the prerequisites in a deliberate order, then at the final step you can
specify a bunch of optional lifecycle methods. Usage looks like this:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">val</span> <span class="pl-c1">MyComponent</span> <span class="pl-k">=</span>
<span class="pl-en">ScalaComponent</span>.builder[<span class="pl-en">Props</span>](<span class="pl-s"><span class="pl-pds">"</span>MyComponent<span class="pl-pds">"</span></span>)
.stateless <span class="pl-c"><span class="pl-c">//</span> step 1</span>
.noBackend <span class="pl-c"><span class="pl-c">//</span> step 2</span>
.render(...) <span class="pl-c"><span class="pl-c">//</span> step 3</span>
.componentWillMount(...) <span class="pl-c"><span class="pl-c">//</span> step 4 (optional)</span>
.componentDidMount(...) <span class="pl-c"><span class="pl-c">//</span> step 4 (optional)</span>
.build <span class="pl-c"><span class="pl-c">//</span> step 4</span></pre></div>
<p>Steps 1 and 2 are optional and are made so in the API via implicits. A minimal example looks like this:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">val</span> <span class="pl-c1">MyComponent</span> <span class="pl-k">=</span>
<span class="pl-en">ScalaComponent</span>.builder[<span class="pl-en">Props</span>](<span class="pl-s"><span class="pl-pds">"</span>MyComponent<span class="pl-pds">"</span></span>)
.render(...) <span class="pl-c"><span class="pl-c">//</span> step 3</span>
.build <span class="pl-c"><span class="pl-c">//</span> step 4</span></pre></div>
<p>Multiple specifications of the same lifecycle method compose.
For example, this is valid and will result in all three procedures executing
at the <code>.componentDidMount</code> lifecycle event.</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">val</span> <span class="pl-c1">MyComponent</span> <span class="pl-k">=</span>
<span class="pl-en">ScalaComponent</span>.builder[<span class="pl-en">Props</span>](<span class="pl-s"><span class="pl-pds">"</span>MyComponent<span class="pl-pds">"</span></span>)
.render(...) <span class="pl-c"><span class="pl-c">//</span> step 3</span>
.componentDidMount(...) <span class="pl-c"><span class="pl-c">//</span> step 4 (optional)</span>
.componentDidMount(...) <span class="pl-c"><span class="pl-c">//</span> step 4 (optional)</span>
.componentDidMount(...) <span class="pl-c"><span class="pl-c">//</span> step 4 (optional)</span>
.build <span class="pl-c"><span class="pl-c">//</span> step 4</span></pre></div>
<h1><a id="user-content-problem" class="anchor" aria-hidden="true" href="#problem"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Problem</h1>
<p>A recent version of React introduced some changes to lifecycle methods
and so I was updating scalajs-react the other day.</p>
<p>The two changes relevant to this article are:</p>
<ol>
<li>A new lifecycle method <code>getSnapshotBeforeUpdate</code> is added, from which you return any arbitrary value called a snapshot.</li>
<li>The lifecycle method <code>componentDidUpdate</code> gets a new parameter which is the value from <code>getSnapshotBeforeUpdate</code> above.</li>
</ol>
<h1><a id="user-content-goals" class="anchor" aria-hidden="true" href="#goals"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Goals</h1>
<p>Let's break down the new React changes into a few rules:</p>
<ol>
<li>The return type of <code>getSnapshotBeforeUpdate</code> needs to match the type of the new <code>componentDidUpdate</code> param.</li>
<li>In the case that <code>getSnapshotBeforeUpdate</code> isn't specified, the type of the new <code>componentDidUpdate</code> param will be <code>Unit</code> (which is <code>undefined</code> in JS).</li>
<li>In order to support composition of multiple <code>getSnapshotBeforeUpdate</code> functions, a means of return value composition (most naturally a <code>Semigroup</code> typeclass) is required.</li>
</ol>
<p>Regarding (3),</p>
<ul>
<li><code>Semigroup</code> would only be required for subsequent calls, not the first, which adds a little complication for values for which <code>Semigroup</code> isn't defined.</li>
<li>scalajs-react doesn't have external dependencies and I don't want to add a <code>Semigroup</code> typeclass to the public API.</li>
<li>I'd be surprised if anyone ever wanted to supply multiple <code>getSnapshotBeforeUpdate</code> functions anyway; if so, one can do it oneself.</li>
</ul>
<p>Therefore I've decided to just not support multiple <code>getSnapshotBeforeUpdate</code> functions. We don't lose parity with React JS anyway.</p>
<p>Let's break down (1) and (2) into more concrete rules:</p>
<ol>
<li><code>getSnapshotBeforeUpdate</code> can only be called 0 or 1 times</li>
<li><code>getSnapshotBeforeUpdate</code> sets the <code>Snapshot</code> type</li>
<li><code>componentDidUpdate</code> receives the <code>Snapshot</code> type</li>
<li>When <code>componentDidUpdate</code> is called and the <code>Snapshot</code> type is undefined, it's set to <code>Unit</code></li>
<li><code>getSnapshotBeforeUpdate</code> cannot occur after <code>componentDidUpdate</code> because it will change the <code>Snapshot</code> type which would invalidate the previous <code>componentDidUpdate</code> where the <code>Snapshot</code> was <code>Unit</code> (and a fn to <code>Unit</code> would be pointless here).</li>
</ol>
<p>Before we continue it's time to emphasise:
type-safety is very important to me.
One of the biggest features of scalajs-react is its strong type-safety.
(As much is reasonable in Scala) if it compiles, I want confidence that it works and is correct.</p>
<p>I want to encode the above rules into the types so that end-users don't have to
read any documentation, have any internal knowledge of these rules, or experience
any runtime exceptions; the compiler should just enforce everything
we discussed such that violations wont even compile.</p>
<h1><a id="user-content-rejected-solution" class="anchor" aria-hidden="true" href="#rejected-solution"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Rejected solution</h1>
<p>Probably the first solution that earlier-me would've reached for, is to create
a new step in the builder API like this:</p>
<pre><code> [STEP 3] [STEP 5]
.render ----(implicit with Snapshot=Unit)---> last step
\ /
\ /
--------> .getSnapshotBeforeUpdate ------> /
[STEP 4]
</code></pre>
<p>There are problems with such a solution:</p>
<ul>
<li>It doesn't scale. If React adds more constraints in future it will become harder to keep a fluent API without introducing unnecessary usage constraints.</li>
<li>External component config fns (<code>LastStep => LastStep</code>) need to be able to configure any part of the lifecycle.</li>
<li><code>ScalaComponent.builder.static</code> is an example where it returns a half-built component allowing further configuration.
It needs to set the <code>shouldComponentUpdate</code> method which would skip step 4 in this approach, or else require that we add nearly everything to both steps (yuk).</li>
</ul>
<h1><a id="user-content-basic-solution" class="anchor" aria-hidden="true" href="#basic-solution"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Basic solution</h1>
<p>Consider this pseudo-code:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">var</span> <span class="pl-smi">snapshotType</span> <span class="pl-k">=</span> <span class="pl-c1">None</span>
<span class="pl-k">def</span> <span class="pl-en">getSnapshotBeforeUpdate</span>[<span class="pl-en">A</span>](<span class="pl-v">f</span>: <span class="pl-en">X</span> <span class="pl-k">=></span> <span class="pl-en">A</span>) <span class="pl-k">=</span> {
snapshotType <span class="pl-k">match</span> {
<span class="pl-k">case</span> <span class="pl-c1">None</span> <span class="pl-k">=></span> snapshotType <span class="pl-k">=</span> <span class="pl-en">Some</span>(<span class="pl-en">A</span>)
<span class="pl-k">case</span> <span class="pl-en">Some</span>(_) <span class="pl-k">=></span> error(<span class="pl-s"><span class="pl-pds">"</span>SnapshotType already defined!<span class="pl-pds">"</span></span>)
}
getSnapshotBeforeUpdate <span class="pl-k">=</span> f
}
<span class="pl-k">def</span> <span class="pl-en">componentDidUpdate</span>(f) <span class="pl-k">=</span> {
snapshotType <span class="pl-k">=</span> <span class="pl-en">Some</span>(snapshotType.getOrElse(<span class="pl-k">Unit</span>))
componentDidUpdate.append(f)
}</pre></div>
<p>We could track this at the term-level at runtime using <code>Option</code> (and typetags).
It's not very type-safe though.
We can still keep the same approach and logic, we just need to lift it up into the type-level so
that it runs at compile-time instead of at runtime.
To do so we'll use a type-level encoding of <code>Option</code>.</p>
<p>This is how you encode <code>Option</code> at the type-level in Scala; first you'll see a term-level equivalent
for contrast:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">object</span> <span class="pl-en">TermLevel</span> {
<span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">Option</span>[<span class="pl-k">+</span><span class="pl-en">A</span>] {
<span class="pl-k">def</span> <span class="pl-en">getOrElse</span>[<span class="pl-en">B</span> <span class="pl-k">></span><span class="pl-k">:</span> <span class="pl-en">A</span>](<span class="pl-v">default</span>: <span class="pl-k">=></span> <span class="pl-en">B</span>)<span class="pl-k">:</span> <span class="pl-en">B</span>
}
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Some</span>[<span class="pl-k">+</span><span class="pl-en">A</span>](<span class="pl-v">value</span>: <span class="pl-en">A</span>) <span class="pl-k">extends</span> <span class="pl-e">Option</span>[<span class="pl-en">A</span>] {
<span class="pl-k">override</span> <span class="pl-k">def</span> <span class="pl-en">getOrElse</span>[<span class="pl-en">B</span> <span class="pl-k">></span><span class="pl-k">:</span> <span class="pl-en">A</span>](<span class="pl-v">default</span>: <span class="pl-k">=></span> <span class="pl-en">B</span>) <span class="pl-k">=</span> value
}
<span class="pl-k">case</span> <span class="pl-k">object</span> <span class="pl-en">None</span> <span class="pl-k">extends</span> <span class="pl-e">Option</span>[<span class="pl-en">Nothing</span>] {
<span class="pl-k">override</span> <span class="pl-k">def</span> <span class="pl-en">getOrElse</span>[<span class="pl-en">B</span> <span class="pl-k">></span><span class="pl-k">:</span> <span class="pl-en">Nothing</span>](<span class="pl-v">default</span>: <span class="pl-k">=></span> <span class="pl-en">B</span>) <span class="pl-k">=</span> default
}
<span class="pl-c"><span class="pl-c">//</span> Example usage</span>
<span class="pl-k">def</span> <span class="pl-en">value</span><span class="pl-k">:</span> <span class="pl-en">Option</span>[<span class="pl-en">Any</span>] <span class="pl-k">=></span> <span class="pl-en">Any</span> <span class="pl-k">=</span> _.getOrElse(())
}
<span class="pl-c"><span class="pl-c">//</span> ===============================================================</span>
<span class="pl-k">object</span> <span class="pl-en">TypeLevel</span> {
<span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">TOption</span> {
<span class="pl-k">type</span> <span class="pl-en">GetOrElse</span>[<span class="pl-en">B</span>]
}
<span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">TSome</span>[<span class="pl-en">A</span>] <span class="pl-k">extends</span> <span class="pl-e">TOption</span> {
<span class="pl-k">override</span> <span class="pl-k">final</span> <span class="pl-k">type</span> <span class="pl-en">GetOrElse</span>[<span class="pl-en">B</span>] <span class="pl-k">=</span> <span class="pl-en">A</span>
}
<span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">TNone</span> <span class="pl-k">extends</span> <span class="pl-e">TOption</span> {
<span class="pl-k">override</span> <span class="pl-k">final</span> <span class="pl-k">type</span> <span class="pl-en">GetOrElse</span>[<span class="pl-en">B</span>] <span class="pl-k">=</span> <span class="pl-en">B</span>
}
<span class="pl-c"><span class="pl-c">//</span> Example usage</span>
<span class="pl-k">type</span> <span class="pl-en">Value</span>[<span class="pl-en">T</span> <span class="pl-k"><</span><span class="pl-k">:</span> <span class="pl-en">TOption</span>] <span class="pl-k">=</span> <span class="pl-en">T</span>#<span class="pl-en">GetOrElse</span>[<span class="pl-k">Unit</span>]
}</pre></div>
<p>Ok, now let's code up a skeleton that will enforce our rules at compile-time:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">final</span> <span class="pl-k">class</span> <span class="pl-en">Builder</span>[<span class="pl-en">SnapshotType</span> <span class="pl-k"><</span><span class="pl-k">:</span> <span class="pl-en">TOption</span>] {
<span class="pl-k">type</span> <span class="pl-en">SnapshotValue</span> <span class="pl-k">=</span> <span class="pl-en">SnapshotType</span>#<span class="pl-en">GetOrElse</span>[<span class="pl-k">Unit</span>]
<span class="pl-k">def</span> <span class="pl-en">getSnapshotBeforeUpdate</span>[<span class="pl-en">A</span>](<span class="pl-v">f</span>: ... <span class="pl-k">=></span> <span class="pl-en">A</span>)
(<span class="pl-k">implicit</span> <span class="pl-v">ev</span>: <span class="pl-en">SnapshotType</span> <span class="pl-k">=</span><span class="pl-k">:</span><span class="pl-k">=</span> <span class="pl-en">TNone</span>)
<span class="pl-k">:</span> <span class="pl-en">Builder</span>[<span class="pl-en">TSome</span>[<span class="pl-en">A</span>]]
<span class="pl-k">def</span> <span class="pl-en">componentDidUpdate</span>(<span class="pl-v">f</span>: <span class="pl-en">SnapshotValue</span> <span class="pl-k">=></span> ...)
<span class="pl-k">:</span> <span class="pl-en">Builder</span>[<span class="pl-en">TSome</span>[<span class="pl-en">SnapshotValue</span>]]
}</pre></div>
<p>Let's compare this to our pseudo-code:</p>
<ol>
<li>The <code>snapshotType</code> var is now a type parameter of <code>Builder</code>.</li>
<li><code>getSnapshotBeforeUpdate</code> would check <code>snapshotType</code> is <code>None</code> and set it to <code>Some(A)</code>.
Now we ask for implicit proof that <code>SnapshotType =:= TNone</code>, set in the return type we can see return a new <code>Builder</code> with <code>SnapshotType</code> set to <code>TSome</code></li>
<li><code>getSnapshotBeforeUpdate</code> throw an error when <code>snapshotType</code> is <code>Some(_)</code>.
Now the compiler will throw an implicit not found error at compile-time when <code>SnapshotType =:= TSome[_]</code>.</li>
<li>Where as before in <code>componentDidUpdate</code> we had <code>snapshotType = Some(snapshotType.getOrElse(Unit))</code>,
we now have the equivalent in that the return type is <code>Builder[TSome[SnapshotValue]]</code>
where <code>SnapshotValue = SnapshotType#GetOrElse[Unit]</code>.</li>
</ol>
<h1><a id="user-content-nice-solution" class="anchor" aria-hidden="true" href="#nice-solution"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Nice solution</h1>
<p>A while back I would've been satisfied with the above solution; it works right?
If you know all of the rules, sure, but the error message to users is going to
be pretty confusing and probably even lead them to think there's some kind of bug in the library.
This is what an error looks like at the moment:</p>
<pre><code>[error] ScalaComponentTest.scala:189: Cannot prove that japgolly.scalajs.react.example.TSome[A] =:= japgolly.scalajs.react.example.TNone.
[error] .getSnapshotBeforeUpdate(???)
[error] ^
[error] one error found
</code></pre>
<p>Nice UX in Scala is a bit of an art; in this case we'll do away with the
generic <code>TOption</code> and create a custom construct for this one specific problem.</p>
<p>First, the new shape. Because this isn't generic anymore we no longer need the
inner type member to be a type constructor which makes usage nicer too
(i.e. <code>T#Value</code> instead of <code>T#GetOrElse[Unit]</code>):</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">UpdateSnapshot</span> {
<span class="pl-k">type</span> <span class="pl-en">Value</span>
}
<span class="pl-k">object</span> <span class="pl-en">UpdateSnapshot</span> {
<span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">None</span> <span class="pl-k">extends</span> <span class="pl-e">UpdateSnapshot</span> {
<span class="pl-k">override</span> <span class="pl-k">final</span> <span class="pl-k">type</span> <span class="pl-en">Value</span> <span class="pl-k">=</span> <span class="pl-k">Unit</span>
}
<span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">Some</span>[<span class="pl-en">A</span>] <span class="pl-k">extends</span> <span class="pl-e">UpdateSnapshot</span> {
<span class="pl-k">override</span> <span class="pl-k">final</span> <span class="pl-k">type</span> <span class="pl-en">Value</span> <span class="pl-k">=</span> <span class="pl-en">A</span>
}
}</pre></div>
<p>Easy enough. Now to improve the UX on failure. First we change the <code>(implicit ev: SnapshotType =:= TNone)</code>
to <code>(implicit ev: UpdateSnapshot.SafetyProof[U])</code> and create:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">object</span> <span class="pl-en">UpdateSnapshot</span> {
<span class="pl-k">@</span>implicitNotFound(<span class="pl-s"><span class="pl-pds">"</span>You can only specify getSnapshotBeforeUpdate once, and it has to be before <span class="pl-pds">"</span></span> <span class="pl-k">+</span>
<span class="pl-s"><span class="pl-pds">"</span>you specify componentDidUpdate, otherwise the snapshot type could become inconsistent.<span class="pl-pds">"</span></span>)
<span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">SafetyProof</span>[<span class="pl-en">U</span> <span class="pl-k"><</span><span class="pl-k">:</span> <span class="pl-en">UpdateSnapshot</span>]
<span class="pl-k">implicit</span> <span class="pl-k">def</span> <span class="pl-en">safetyProof</span>[<span class="pl-en">U</span> <span class="pl-k"><</span><span class="pl-k">:</span> <span class="pl-en">UpdateSnapshot</span>](<span class="pl-k">implicit</span> <span class="pl-v">ev</span>: <span class="pl-en">U</span> <span class="pl-k">=</span><span class="pl-k">:</span><span class="pl-k">=</span> <span class="pl-en">UpdateSnapshot</span>.<span class="pl-c1">None</span>)<span class="pl-k">:</span> <span class="pl-en">SafetyProof</span>[<span class="pl-en">U</span>] <span class="pl-k">=</span>
<span class="pl-c1">null</span>.<span class="pl-c1">asInstanceOf</span>[<span class="pl-en">SafetyProof</span>[<span class="pl-en">U</span>]]
}</pre></div>
<p>The <code>(implicit ev: U =:= UpdateSnapshot.None)</code> is still part of the solution, but this time it's indirect.
It's a dependency on the availability of implicit <code>SafetyProof</code>. Thus the logic is still the same,
just users will never see it as an error message.</p>
<p>The <code>@implicitNotFound</code> annotation on <code>SafetyProof</code> is the pudding.
It will cause our custom error message to be displayed
as a compilation error when someone breaks the rules.</p>
<p>Using <code>null.asInstanceOf[SafetyProof[U]]</code> is a performance optimisation;
<code>new SafetyProof[U]{}</code> is fine too but I'd prefer to avoid the allocation
and more importantly, by never actually creating or using <code>SafetyProof</code> it can be
completely elided from Scala.JS output which means a smaller download for your
webapp's end-users.</p>
<p>Finally, our new builder excerpt looks like this:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">final</span> <span class="pl-k">class</span> <span class="pl-en">Builder</span>[<span class="pl-en">U</span> <span class="pl-k"><</span><span class="pl-k">:</span> <span class="pl-en">UpdateSnapshot</span>] {
<span class="pl-k">type</span> <span class="pl-en">SnapshotValue</span> <span class="pl-k">=</span> <span class="pl-en">U</span>#<span class="pl-en">Value</span>
<span class="pl-k">def</span> <span class="pl-en">getSnapshotBeforeUpdate</span>[<span class="pl-en">A</span>](<span class="pl-v">f</span>: ... <span class="pl-k">=></span> <span class="pl-en">A</span>)
(<span class="pl-k">implicit</span> <span class="pl-v">ev</span>: <span class="pl-en">UpdateSnapshot</span>.<span class="pl-en">SafetyProof</span>[<span class="pl-en">U</span>])
<span class="pl-k">:</span> <span class="pl-en">Builder</span>[<span class="pl-en">UpdateSnapshot</span>.<span class="pl-en">Some</span>[<span class="pl-en">A</span>]]
<span class="pl-k">def</span> <span class="pl-en">componentDidUpdate</span>(<span class="pl-v">f</span>: <span class="pl-en">SnapshotValue</span> <span class="pl-k">=></span> ...)
<span class="pl-k">:</span> <span class="pl-en">Builder</span>[<span class="pl-en">UpdateSnapshot</span>.<span class="pl-en">Some</span>[<span class="pl-en">SnapshotValue</span>]]
}</pre></div>
<p>And let's look at what errors look like now:</p>
<pre><code>[error] ScalaComponentTest.scala:189: You can only specify getSnapshotBeforeUpdate once, and it has to be before you specify componentDidUpdate, otherwise the snapshot type could become inconsistent.
[error] .getSnapshotBeforeUpdate(???)
[error] ^
[error] one error found
</code></pre>
<h1><a id="user-content-done" class="anchor" aria-hidden="true" href="#done"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Done</h1>
<p>That's all. I hope you've enjoyed.
If you're interested, the full patch that went into scalajs-react is here:</p>
<p><a href="https://github.com/japgolly/scalajs-react/commit/ee81acf12c1039997460a7cac3d759fda6577533">https://github.com/japgolly/scalajs-react/commit/ee81acf12c1039997460a7cac3d759fda6577533</a></p>
</article>
</div>
</div>
</div>
</div>
</div>
</div>
David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com0tag:blogger.com,1999:blog-5837865295676240265.post-3566891574353854932017-12-13T12:09:00.000+11:002017-12-13T12:10:57.336+11:00Practical Awesome Recursion - Ch 02: Catamorphisms<style>
.goost{font-size:16px;color:#333;text-align:left;/*!
* GitHub Light v0.4.1
* Copyright (c) 2012 - 2017 GitHub, Inc.
* Licensed under MIT (https://github.com/primer/github-syntax-theme-generator/blob/master/LICENSE)
*/direction:ltr}.goost .markdown-body{font-family:-apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol";font-size:16px;line-height:1.5;word-wrap:break-word}.goost .markdown-body::before{display:table;content:""}.goost .markdown-body::after{display:table;clear:both;content:""}.goost .markdown-body>*:first-child{margin-top:0 !important}.goost .markdown-body>*:last-child{margin-bottom:0 !important}.goost .markdown-body a:not([href]){color:inherit;text-decoration:none}.goost .markdown-body .absent{color:#cb2431}.goost .markdown-body .anchor{float:left;padding-right:4px;margin-left:-20px;line-height:1}.goost .markdown-body .anchor:focus{outline:none}.goost .markdown-body p,.goost .markdown-body blockquote,.goost .markdown-body ul,.goost .markdown-body ol,.goost .markdown-body dl,.goost .markdown-body table,.goost .markdown-body pre{margin-top:0;margin-bottom:16px}.goost .markdown-body hr{height:0.25em;padding:0;margin:24px 0;background-color:#e1e4e8;border:0}.goost .markdown-body blockquote{padding:0 1em;color:#6a737d;border-left:0.25em solid #dfe2e5}.goost .markdown-body blockquote>:first-child{margin-top:0}.goost .markdown-body blockquote>:last-child{margin-bottom:0}.goost .markdown-body kbd{display:inline-block;padding:3px 5px;font-size:11px;line-height:10px;color:#444d56;vertical-align:middle;background-color:#fafbfc;border:solid 1px #c6cbd1;border-bottom-color:#959da5;border-radius:3px;box-shadow:inset 0 -1px 0 #959da5}.goost .markdown-body h1,.goost .markdown-body h2,.goost .markdown-body h3,.goost .markdown-body h4,.goost .markdown-body h5,.goost .markdown-body h6{margin-top:24px;margin-bottom:16px;font-weight:600;line-height:1.25}.goost .markdown-body h1 .octicon-link,.goost .markdown-body h2 .octicon-link,.goost .markdown-body h3 .octicon-link,.goost .markdown-body h4 .octicon-link,.goost .markdown-body h5 .octicon-link,.goost .markdown-body h6 .octicon-link{color:#1b1f23;vertical-align:middle;visibility:hidden}.goost .markdown-body h1:hover .anchor,.goost .markdown-body h2:hover .anchor,.goost .markdown-body h3:hover .anchor,.goost .markdown-body h4:hover .anchor,.goost .markdown-body h5:hover .anchor,.goost .markdown-body h6:hover .anchor{text-decoration:none}.goost .markdown-body h1:hover .anchor .octicon-link,.goost .markdown-body h2:hover .anchor .octicon-link,.goost .markdown-body h3:hover .anchor .octicon-link,.goost .markdown-body h4:hover .anchor .octicon-link,.goost .markdown-body h5:hover .anchor .octicon-link,.goost .markdown-body h6:hover .anchor .octicon-link{visibility:visible}.goost .markdown-body h1 tt,.goost .markdown-body h1 code,.goost .markdown-body h2 tt,.goost .markdown-body h2 code,.goost .markdown-body h3 tt,.goost .markdown-body h3 code,.goost .markdown-body h4 tt,.goost .markdown-body h4 code,.goost .markdown-body h5 tt,.goost .markdown-body h5 code,.goost .markdown-body h6 tt,.goost .markdown-body h6 code{font-size:inherit}.goost .markdown-body h1{padding-bottom:0.3em;font-size:2em;border-bottom:1px solid #eaecef}.goost .markdown-body h2{padding-bottom:0.3em;font-size:1.5em;border-bottom:1px solid #eaecef}.goost .markdown-body h3{font-size:1.25em}.goost .markdown-body h4{font-size:1em}.goost .markdown-body h5{font-size:0.875em}.goost .markdown-body h6{font-size:0.85em;color:#6a737d}.goost .markdown-body ul,.goost .markdown-body ol{padding-left:2em}.goost .markdown-body ul.no-list,.goost .markdown-body ol.no-list{padding:0;list-style-type:none}.goost .markdown-body ul ul,.goost .markdown-body ul ol,.goost .markdown-body ol ol,.goost .markdown-body ol ul{margin-top:0;margin-bottom:0}.goost .markdown-body li{word-wrap:break-all}.goost .markdown-body li>p{margin-top:16px}.goost .markdown-body li+li{margin-top:0.25em}.goost .markdown-body dl{padding:0}.goost .markdown-body dl dt{padding:0;margin-top:16px;font-size:1em;font-style:italic;font-weight:600}.goost .markdown-body dl dd{padding:0 16px;margin-bottom:16px}.goost .markdown-body table{display:block;width:100%;overflow:auto}.goost .markdown-body table th{font-weight:600}.goost .markdown-body table th,.goost .markdown-body table td{padding:6px 13px;border:1px solid #dfe2e5}.goost .markdown-body table tr{background-color:#fff;border-top:1px solid #c6cbd1}.goost .markdown-body table tr:nth-child(2n){background-color:#f6f8fa}.goost .markdown-body table img{background-color:transparent}.goost .markdown-body img{max-width:100%;box-sizing:content-box;background-color:#fff}.goost .markdown-body img[align=right]{padding-left:20px}.goost .markdown-body img[align=left]{padding-right:20px}.goost .markdown-body .emoji{max-width:none;vertical-align:text-top;background-color:transparent}.goost .markdown-body span.frame{display:block;overflow:hidden}.goost .markdown-body span.frame>span{display:block;float:left;width:auto;padding:7px;margin:13px 0 0;overflow:hidden;border:1px solid #dfe2e5}.goost .markdown-body span.frame span img{display:block;float:left}.goost .markdown-body span.frame span span{display:block;padding:5px 0 0;clear:both;color:#24292e}.goost .markdown-body span.align-center{display:block;overflow:hidden;clear:both}.goost .markdown-body span.align-center>span{display:block;margin:13px auto 0;overflow:hidden;text-align:center}.goost .markdown-body span.align-center span img{margin:0 auto;text-align:center}.goost .markdown-body span.align-right{display:block;overflow:hidden;clear:both}.goost .markdown-body span.align-right>span{display:block;margin:13px 0 0;overflow:hidden;text-align:right}.goost .markdown-body span.align-right span img{margin:0;text-align:right}.goost .markdown-body span.float-left{display:block;float:left;margin-right:13px;overflow:hidden}.goost .markdown-body span.float-left span{margin:13px 0 0}.goost .markdown-body span.float-right{display:block;float:right;margin-left:13px;overflow:hidden}.goost .markdown-body span.float-right>span{display:block;margin:13px auto 0;overflow:hidden;text-align:right}.goost .markdown-body code,.goost .markdown-body tt{padding:0.2em 0.4em;margin:0;font-size:85%;background-color:rgba(27,31,35,0.05);border-radius:3px}.goost .markdown-body code br,.goost .markdown-body tt br{display:none}.goost .markdown-body del code{text-decoration:inherit}.goost .markdown-body pre{word-wrap:normal}.goost .markdown-body pre>code{padding:0;margin:0;font-size:100%;word-break:normal;white-space:pre;background:transparent;border:0}.goost .markdown-body .highlight{margin-bottom:16px}.goost .markdown-body .highlight pre{margin-bottom:0;word-break:normal}.goost .markdown-body .highlight pre,.goost .markdown-body pre{padding:16px;overflow:auto;font-size:85%;line-height:1.45;background-color:#f6f8fa;border-radius:3px}.goost .markdown-body pre code,.goost .markdown-body pre tt{display:inline;max-width:auto;padding:0;margin:0;overflow:visible;line-height:inherit;word-wrap:normal;background-color:transparent;border:0}.goost .markdown-body .csv-data td,.goost .markdown-body .csv-data th{padding:5px;overflow:hidden;font-size:12px;line-height:1;text-align:left;white-space:nowrap}.goost .markdown-body .csv-data .blob-num{padding:10px 8px 9px;text-align:right;background:#fff;border:0}.goost .markdown-body .csv-data tr{border-top:0}.goost .markdown-body .csv-data th{font-weight:600;background:#f6f8fa;border-top:0}.goost .pl-c{color:#6a737d}.goost .pl-c1,.goost .pl-s .pl-v{color:#005cc5}.goost .pl-e,.goost .pl-en{color:#6f42c1}.goost .pl-smi,.goost .pl-s .pl-s1{color:#24292e}.goost .pl-ent{color:#22863a}.goost .pl-k{color:#d73a49}.goost .pl-s,.goost .pl-pds,.goost .pl-s .pl-pse .pl-s1,.goost .pl-sr,.goost .pl-sr .pl-cce,.goost .pl-sr .pl-sre,.goost .pl-sr .pl-sra{color:#032f62}.goost .pl-v,.goost .pl-smw{color:#e36209}.goost .pl-bu{color:#b31d28}.goost .pl-ii{color:#fafbfc;background-color:#b31d28}.goost .pl-c2{color:#fafbfc;background-color:#d73a49}.goost .pl-c2::before{content:"^M"}.goost .pl-sr .pl-cce{font-weight:bold;color:#22863a}.goost .pl-ml{color:#735c0f}.goost .pl-mh,.goost .pl-mh .pl-en,.goost .pl-ms{font-weight:bold;color:#005cc5}.goost .pl-mi{font-style:italic;color:#24292e}.goost .pl-mb{font-weight:bold;color:#24292e}.goost .pl-md{color:#b31d28;background-color:#ffeef0}.goost .pl-mi1{color:#22863a;background-color:#f0fff4}.goost .pl-mc{color:#e36209;background-color:#ffebda}.goost .pl-mi2{color:#f6f8fa;background-color:#005cc5}.goost .pl-mdr{font-weight:bold;color:#6f42c1}.goost .pl-ba{color:#586069}.goost .pl-sg{color:#959da5}.goost .pl-corl{text-decoration:underline;color:#032f62}.goost .breadcrumb{margin-bottom:10px;font-size:18px;color:#586069}.goost .breadcrumb .separator::before,.goost .breadcrumb .separator::after{content:" "}.goost .breadcrumb strong.final-path{color:#24292e}.goost .breadcrumb .zeroclipboard-button{display:inline-block;margin-left:5px}.goost .breadcrumb .repo-root{font-weight:600}.goost .breadcrumb .octicon{vertical-align:-2px}.goost .editor-license-template,.goost .editor-code-of-conduct-template,.goost .editor-gitignore-template{position:relative;top:3px;display:block;float:right;font-size:14px}.goost .editor-license-template .select-menu-git-ignore,.goost .editor-code-of-conduct-template .select-menu-git-ignore,.goost .editor-gitignore-template .select-menu-git-ignore{right:0}.goost .editor-abort{display:inline;font-size:14px}.goost .blob-interaction-bar{position:relative;background-color:#f2f2f2;border-bottom:1px solid #e5e5e5}.goost .blob-interaction-bar::before{display:table;content:""}.goost .blob-interaction-bar::after{display:table;clear:both;content:""}.goost .blob-interaction-bar .octicon-search{position:absolute;top:10px;left:10px;font-size:12px;color:#586069}.goost .blob-filter{width:100%;padding:4px 20px 5px 30px;font-size:12px;border:0;border-radius:0;outline:none}.goost .blob-filter:focus{outline:none}.goost .html-blob{margin-bottom:15px}.goost .TagsearchPopover-content{width:inherit;max-width:600px}.goost .license-summary-octicon{color:#959da5}.goost .rule-type-permissions{color:#28a745}.goost .rule-type-conditions{color:#0366d6}.goost .rule-type-limitations{color:#d73a49}.goost .blob-wrapper{overflow-x:auto;overflow-y:hidden;border-bottom-right-radius:3px;border-bottom-left-radius:3px}.goost .blob-wrapper-embedded{max-height:240px;overflow-y:auto}.goost .diff-table{width:100%;border-collapse:separate}.goost .diff-table .line-comments{padding:10px;vertical-align:top;border-top:1px solid #e1e4e8}.goost .diff-table .line-comments:first-child+.empty-cell{border-left-width:1px}.goost .diff-table tr:not(:last-child) .line-comments{border-top:1px solid #e1e4e8;border-bottom:1px solid #e1e4e8}.goost .blob-num{width:1%;min-width:50px;padding-right:10px;padding-left:10px;font-family:"SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace;font-size:12px;line-height:20px;color:rgba(27,31,35,0.3);text-align:right;white-space:nowrap;vertical-align:top;cursor:pointer;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}.goost .blob-num:hover{color:rgba(27,31,35,0.6)}.goost .blob-num::before{content:attr(data-line-number)}.goost .blob-num.non-expandable{cursor:default}.goost .blob-num.non-expandable:hover{color:rgba(27,31,35,0.3)}.goost .blob-code{position:relative;padding-right:10px;padding-left:10px;line-height:20px;vertical-align:top}.goost .blob-code-inner{overflow:visible;font-family:"SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace;font-size:12px;color:#24292e;word-wrap:normal;white-space:pre}.goost .blob-code-inner .x-first{border-top-left-radius:0.2em;border-bottom-left-radius:0.2em}.goost .blob-code-inner .x-last{border-top-right-radius:0.2em;border-bottom-right-radius:0.2em}.goost .blob-code-inner::before{content:""}.goost .blob-code-inner.highlighted{background-color:#fffbdd}.goost .soft-wrap .diff-table{table-layout:fixed}.goost .soft-wrap .blob-code{padding-left:18px;text-indent:-7px}.goost .soft-wrap .blob-code-inner{word-wrap:break-word;white-space:pre-wrap}.goost .soft-wrap .no-nl-marker{display:none}.goost .soft-wrap .add-line-comment{margin-left:-28px}.goost .blob-num-hunk,.goost .blob-code-hunk,.goost .blob-num-expandable,.goost .blob-code-expandable{color:rgba(27,31,35,0.5);vertical-align:middle}.goost .blob-num-hunk,.goost .blob-num-expandable{background-color:#dbedff}.goost .blob-code-hunk,.goost .blob-code-expandable{padding-top:4px;padding-bottom:4px;background-color:#f1f8ff;border-width:1px 0}.goost .blob-expanded .blob-num,.goost .blob-expanded .blob-code{background-color:#fafbfc}.goost .blob-expanded+tr:not(.blob-expanded) .blob-num,.goost .blob-expanded+tr:not(.blob-expanded) .blob-code{border-top:1px solid #eaecef}.goost .blob-expanded .blob-num-hunk{border-top:1px solid #eaecef}.goost tr:not(.blob-expanded)+.blob-expanded .blob-num,.goost tr:not(.blob-expanded)+.blob-expanded .blob-code{border-top:1px solid #eaecef}.goost .blob-num-expandable{padding:0;font-size:12px;text-align:center}.goost .blob-num-expandable .octicon{vertical-align:top}.goost .blob-num-expandable .diff-expander{display:block;width:auto;height:auto;padding:4px 11px 4px 10px;margin-right:-1px;color:#586069;cursor:pointer}.goost .blob-num-expandable .diff-expander:hover{color:#fff;text-shadow:none;background-color:#0366d6;border-color:#0366d6}.goost .blob-code-addition{background-color:#e6ffed}.goost .blob-code-addition .x{color:#24292e;background-color:#acf2bd}.goost .blob-num-addition{background-color:#cdffd8;border-color:#bef5cb}.goost .blob-code-deletion{background-color:#ffeef0}.goost .blob-code-deletion .x{color:#24292e;background-color:#fdb8c0}.goost .blob-num-deletion{background-color:#ffdce0;border-color:#fdaeb7}.goost .selected-line.blob-code{background-color:#fffbdd}.goost .selected-line.blob-code .x{background-color:transparent}.goost .selected-line.blob-num{background-color:#fff5b1;border-color:#ffea7f}.goost .add-line-comment{position:relative;z-index:5;float:left;width:22px;height:22px;margin:-2px -10px -2px -20px;line-height:21px;color:#fff;text-align:center;text-indent:0;cursor:pointer;background-color:#0366d6;background-image:linear-gradient(#0372ef, #0366d6);border-radius:3px;box-shadow:0 1px 4px rgba(27,31,35,0.15);opacity:0;transition:-webkit-transform 0.1s ease-in-out;transition:transform 0.1s ease-in-out;-webkit-transform:scale(0.8, 0.8);transform:scale(0.8, 0.8)}.goost .add-line-comment:hover{-webkit-transform:scale(1, 1);transform:scale(1, 1)}.is-hovered .goost .add-line-comment,.goost .add-line-comment:focus{opacity:1}.goost .add-line-comment .octicon{vertical-align:text-top;pointer-events:none}.goost .add-line-comment.octicon-check{background:#333;opacity:1}.goost .inline-comment-form{border:1px solid #dfe2e5;border-radius:3px}.goost .inline-review-comment{margin-top:0 !important;margin-bottom:10px !important}.goost .inline-review-comment .gc:first-child+tr .blob-num,.goost .inline-review-comment .gc:first-child+tr .blob-code{padding-top:5px}.goost .inline-review-comment tr:last-child{border-bottom-right-radius:2px;border-bottom-left-radius:2px}.goost .inline-review-comment tr:last-child .blob-num,.goost .inline-review-comment tr:last-child .blob-code{padding-bottom:8px}.goost .inline-review-comment tr:last-child .blob-num:first-child,.goost .inline-review-comment tr:last-child .blob-code:first-child{border-bottom-left-radius:2px}.goost .inline-review-comment tr:last-child .blob-num:last-child,.goost .inline-review-comment tr:last-child .blob-code:last-child{border-bottom-right-radius:2px}.goost .timeline-inline-comments{width:100%;table-layout:fixed}.goost .timeline-inline-comments .inline-comments,.goost .show-inline-notes .inline-comments{display:table-row}.goost .inline-comments{display:none}.goost .inline-comments.is-collapsed{display:none}.goost .inline-comments .line-comments.is-collapsed{visibility:hidden}.goost .inline-comments .line-comments+.blob-num{border-left-width:1px}.goost .inline-comments .timeline-comment{margin-bottom:10px}.goost .inline-comments .inline-comment-form,.goost .inline-comments .inline-comment-form-container{max-width:780px}.goost .comment-holder{max-width:780px}.goost .line-comments+.line-comments,.goost .empty-cell+.line-comments{border-left:1px solid #eaecef}.goost .inline-comment-form-container .inline-comment-form,.goost .inline-comment-form-container.open .inline-comment-form-actions{display:none}.goost .inline-comment-form-container .inline-comment-form-actions,.goost .inline-comment-form-container.open .inline-comment-form{display:block}.goost body.split-diff .container,.goost body.full-width .container{width:100%;padding-right:20px;padding-left:20px}.goost body.split-diff .repository-content,.goost body.full-width .repository-content{width:100%}.goost body.split-diff .new-pr-form,.goost body.full-width .new-pr-form{max-width:980px}.goost body.split-diff .new-pr-form .discussion-sidebar,.goost body.full-width .new-pr-form .discussion-sidebar{width:200px}.goost .file-diff-split{table-layout:fixed}.goost .file-diff-split .blob-code+.blob-num{border-left:1px solid #f6f8fa}.goost .file-diff-split .blob-code-inner{word-wrap:break-word;white-space:pre-wrap}.goost .file-diff-split .empty-cell{cursor:default;background-color:#fafbfc;border-right-color:#eaecef}.goost .submodule-diff-stats .octicon-diff-removed{color:#cb2431}.goost .submodule-diff-stats .octicon-diff-renamed{color:#677a85}.goost .submodule-diff-stats .octicon-diff-modified{color:#d0b44c}.goost .submodule-diff-stats .octicon-diff-added{color:#28a745}.goost .BlobToolbar{left:-17px}.goost .BlobToolbar-dropdown{margin-left:-2px}.goost .task-list-item{list-style-type:none}.goost .task-list-item label{font-weight:normal}.goost .task-list-item.enabled label{cursor:pointer}.goost .task-list-item+.task-list-item{margin-top:3px}.goost .task-list-item .handle{display:none}.goost .task-list-item-checkbox{margin:0 0.2em 0.25em -1.6em;vertical-align:middle}.goost .reorderable-task-lists .markdown-body .contains-task-list{padding:0}.goost .reorderable-task-lists .markdown-body li:not(.task-list-item){margin-left:26px}.goost .reorderable-task-lists .markdown-body ol:not(.contains-task-list) li,.goost .reorderable-task-lists .markdown-body ul:not(.contains-task-list) li{margin-left:0}.goost .reorderable-task-lists .markdown-body li p{margin-top:0}.goost .reorderable-task-lists .markdown-body .task-list-item{padding-right:15px;padding-left:42px;margin-right:-15px;margin-left:-15px;border:1px solid transparent}.goost .reorderable-task-lists .markdown-body .task-list-item+.task-list-item{margin-top:0}.goost .reorderable-task-lists .markdown-body .task-list-item .contains-task-list{padding-top:4px}.goost .reorderable-task-lists .markdown-body .task-list-item .handle{display:block;float:left;width:20px;padding:2px 0 0 2px;margin-left:-43px;opacity:0}.goost .reorderable-task-lists .markdown-body .task-list-item .drag-handle{fill:#333}.goost .reorderable-task-lists .markdown-body .task-list-item.hovered{background:#fafafa;border-top-color:#ededed;border-bottom-color:#ededed}.goost .reorderable-task-lists .markdown-body .task-list-item.hovered>.handle{opacity:1}.goost .reorderable-task-lists .markdown-body .task-list-item.is-dragging{opacity:0}.goost .reorderable-task-lists .markdown-body .task-list-item.is-ghost{border-right-color:#ededed;border-left-color:#ededed}.goost .review-comment-contents .markdown-body .task-list-item{padding-left:42px;margin-right:-12px;margin-left:-12px;border-top-left-radius:3px;border-bottom-left-radius:3px}.goost .review-comment-contents .markdown-body .task-list-item.hovered{border-left-color:#ededed}.goost .highlight{padding:0;margin:0;font-family:"SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace;font-size:12px;font-weight:normal;line-height:1.4;color:#333;background:#fff;border:0}.goost .render-viewer-error,.goost .render-viewer-fatal,.goost .render-viewer-invalid,.goost .octospinner{display:none}.goost iframe.render-viewer{width:100%;height:480px;overflow:hidden;border:0}.goost pre,.goost code{font-family:"SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace !important;white-space:pre}.goost .goost-meta{padding:10px;overflow:hidden;font:12px -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol";color:#586069;background-color:#f7f7f7;border-radius:0 0 2px 2px}.goost .goost-meta a{font-weight:600;color:#666;text-decoration:none;border:0}.goost .goost-data{overflow:auto;word-wrap:normal;background-color:#fff;border-bottom:1px solid #ddd;border-radius:2px 2px 0 0}.goost .goost-file{margin-bottom:1em;font-family:"SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace;border:1px solid #ddd;border-bottom:1px solid #ccc;border-radius:3px}.goost .goost-file article{padding:6px}.goost .goost-file .scroll .goost-data{position:absolute;top:0;right:0;bottom:30px;left:0;overflow:scroll}.goost .goost-file .scroll .goost-meta{position:absolute;right:0;bottom:0;left:0}.goost .blob-num{min-width:inherit;padding:1px 10px !important;background:transparent}.goost .blob-code{padding:1px 10px !important;text-align:left;background:transparent;border:0}.goost .blob-wrapper table{border-collapse:collapse}.goost .blob-wrapper tr:first-child td{padding-top:4px}.goost .markdown-body .anchor{display:none}
.goost .goost-file {border:none}
.goost .goost-data {border-bottom:none}
.goost .markdown-body .highlight pre, .goost .markdown-body pre {font-size:0.85rem}
</style>
<div id="goost84186183" class="goost">
<div class="goost-file">
<div class="goost-data">
<div class="js-goost-file-update-container js-task-list-container file-box">
<div id="file-blog-md" class="file">
<div id="readme" class="readme blob instapaper_body">
<article class="markdown-body entry-content" itemprop="text"><p>Recursion schemes are awesome, and practical.
This is chapter 2 in a <a href="https://github.com/japgolly/learning/tree/public/recursion-blog">series of blog posts about recursion schemes</a>.
This series uses Scala, and will focus on usage and applicability.
It will be scarce in theory, and abundant in examples.
The theory is valuable and fascinating, but I often find that knowing the theory alone is only half understanding.
The other half of understanding comes from, and enables, practical application.
I recommend you bounce back and forth between this series and theory.
The internet is rich with blogs and videos on theory that explain it better than I would, so use those awesome resources.</p>
<h2><a href="#before-you-start" aria-hidden="true" class="anchor" id="user-content-before-you-start"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Before you start...</h2>
<p>If you don't know what I mean by any of the following,
read or skim <a href="https://japgolly.blogspot.com/2017/11/practical-awesome-recursion-ch-01.html" rel="nofollow">chapter 1 of this series</a>.</p>
<ul>
<li><code>Fix[_]</code></li>
<li><code>IntList</code> / <code>IntListF[_]</code></li>
<li><code>BinaryTree</code> / <code>BinaryTreeF[_]</code></li>
</ul>
<p>Also, this series doesn't depend on, or emphasise, Matryoshka or any other library.
If you'd like to understand why, I've explained <a href="https://github.com/japgolly/learning/blob/public/recursion-blog/FAQ.md">here in the FAQ</a>.</p>
<h1><a href="#the-catamorphism" aria-hidden="true" class="anchor" id="user-content-the-catamorphism"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>The Catamorphism</h1>
<p>What is a catamorphism?
It can be answered from a few perspectives.</p>
<h3><a href="#what" aria-hidden="true" class="anchor" id="user-content-what"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>What</h3>
<p>It's often referred to as a fold over your data structure.
Examples that sum or count values are very common.
Conceptually speaking, an example using Scala stdlib would be
<code>List(1, 3, 7).foldLeft(0)(_ + _) == 11</code>.
As you'll see, folds and catamorphisms are capable of much more than calculating numbers.</p>
<p>The definition of catamorphism is:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">def</span> <span class="pl-en">cata</span>[<span class="pl-en">F</span>[_]<span class="pl-k">:</span> <span class="pl-en">Functor</span>, <span class="pl-en">A</span>, <span class="pl-en">B</span>](<span class="pl-v">fAlgebra</span>: <span class="pl-en">F</span>[<span class="pl-en">A</span>] <span class="pl-k">=></span> <span class="pl-en">A</span>)(<span class="pl-v">f</span>: <span class="pl-en">Fix</span>[<span class="pl-en">F</span>])<span class="pl-k">:</span> <span class="pl-en">A</span> <span class="pl-k">=</span>
fAlgebra(f.unfix.map(cata(fAlgebra)))</pre></div>
<p>The first argument <code>fAlgebra</code> is so-called because <code>F[A] => A</code> is known as an
F-algebra. <em>What</em> it is, is your folding logic.
You implement your folding logic as a function that processes a single
level/layer of your structure without recursion.
When I say level/layer, I mean in terms of recursive depth, example:</p>
<pre><code>IntList
=======
This is equivalent to List(1, 2, 3, 4, 5).
IntCons(1, _) ← Level 1
IntCons(2, _) ← Level 2
IntCons(3, _) ← Level 3
IntCons(4, _) ← Level 4
IntCons(5, _) ← Level 5
IntNil ← Level 6
</code></pre>
<pre><code>BinaryTree
==========
Branch(_, "root", _) ← Level 1
|
+-----------+-----------+
| |
Leaf Branch(_, "right", _) ← Level 2
| |
Leaf Leaf ← Level 3
</code></pre>
<h3><a href="#how" aria-hidden="true" class="anchor" id="user-content-how"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>How</h3>
<p>Look at the definition of <code>cata</code> and at the shape/type of the f-algebra.
There's an interesting note about parametricity. The only means <code>cata</code>
has of producing an <code>A</code> is to call <code>fAlgebra</code>... which requires an <code>F[A]</code>...
so how does it put an <code>A</code> in the <code>F[_]</code> to call the function,
if it can't produce <code>A</code>s otherwise?
Remember that the hole in <code>F[_]</code> represents the recursive case?
In non-recursive cases (eg. <code>Nil</code> in a cons list) the type is a phantom-type,
completely unused, or covariant and <code>Nothing</code>. For example:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Eg</span>[<span class="pl-en">T</span>](<span class="pl-v">int</span>: <span class="pl-k">Int</span>)
<span class="pl-k">def</span> <span class="pl-en">changeIt</span>[<span class="pl-en">X</span>, <span class="pl-en">Y</span>](<span class="pl-v">eg</span>: <span class="pl-en">Eg</span>[<span class="pl-en">X</span>])<span class="pl-k">:</span> <span class="pl-en">Eg</span>[<span class="pl-en">Y</span>] <span class="pl-k">=</span>
<span class="pl-en">Eg</span>(eg.int)</pre></div>
<p>When a type variable is unused you can replace it with anything you want,
which is exactly what happens in <code>Functor[F]</code>.</p>
<p>It's also important to understand the order in which things happen.
Catamorphisms:</p>
<ol>
<li>start at the root (their input, the <code>f: Fix[F]</code>)</li>
<li>(computationally) move to the leaves</li>
<li>calculate their way back to the root</li>
</ol>
<p>Which means your folding logic is going to start executing against all the leaves
first, then their parents, then <em>their</em> parents, etc.</p>
<h3><a href="#optimisation" aria-hidden="true" class="anchor" id="user-content-optimisation"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Optimisation</h3>
<p>I did say that this is a practical series.
This is a good a place as any to mention that this <code>cata</code> definition,
while correct, is inefficient.</p>
<p>Every time it recurses it has to create the same functions with the same logic
over and over again. We can make it more efficient by creating what we need once
and reusing it.</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">def</span> <span class="pl-en">cata2</span>[<span class="pl-en">F</span>[_], <span class="pl-en">A</span>, <span class="pl-en">B</span>](<span class="pl-v">algebra</span>: <span class="pl-en">F</span>[<span class="pl-en">A</span>] <span class="pl-k">=></span> <span class="pl-en">A</span>)(<span class="pl-v">f</span>: <span class="pl-en">Fix</span>[<span class="pl-en">F</span>])(<span class="pl-k">implicit</span> <span class="pl-en">F</span><span class="pl-k">:</span> <span class="pl-en">Functor</span>[<span class="pl-en">F</span>])<span class="pl-k">:</span> <span class="pl-en">A</span> <span class="pl-k">=</span> {
<span class="pl-k">var</span> <span class="pl-en">self</span><span class="pl-k">:</span> <span class="pl-en">Fix</span>[<span class="pl-en">F</span>] <span class="pl-k">=></span> <span class="pl-en">A</span> <span class="pl-k">=</span> <span class="pl-c1">null</span>
<span class="pl-c1">self</span> <span class="pl-k">=</span> f <span class="pl-k">=></span> algebra(<span class="pl-en">F</span>.map(f.unfix)(<span class="pl-c1">self</span>))
<span class="pl-c1">self</span>(f)
}</pre></div>
<p>In this new definition, we create the recursive function once and reuse it.
Despite the null, it's 100% safe because we (provably) set it before it's used.</p>
<p>Let's measure it. How fast does it perform in comparison to the original?
105%? 110%?</p>
<div class="highlight highlight-source-scala"><pre>[info] <span class="pl-en">Benchmark</span> (size) <span class="pl-en">Mode</span> <span class="pl-en">Cnt</span> <span class="pl-en">Score</span> <span class="pl-en">Error</span> <span class="pl-en">Units</span>
[info] <span class="pl-en">RecursionBM</span>.cata <span class="pl-c1">10</span> avgt <span class="pl-c1">10</span> <span class="pl-c1">0.274</span> ± <span class="pl-c1">0.003</span> us<span class="pl-k">/</span>op
[info] <span class="pl-en">RecursionBM</span>.cata2 <span class="pl-c1">10</span> avgt <span class="pl-c1">10</span> <span class="pl-c1">0.146</span> ± <span class="pl-c1">0.001</span> us<span class="pl-k">/</span>op
[info] <span class="pl-en">RecursionBM</span>.cata <span class="pl-c1">100</span> avgt <span class="pl-c1">10</span> <span class="pl-c1">2.323</span> ± <span class="pl-c1">0.020</span> us<span class="pl-k">/</span>op
[info] <span class="pl-en">RecursionBM</span>.cata2 <span class="pl-c1">100</span> avgt <span class="pl-c1">10</span> <span class="pl-c1">1.555</span> ± <span class="pl-c1">0.006</span> us<span class="pl-k">/</span>op
[info] <span class="pl-en">RecursionBM</span>.cata <span class="pl-c1">1000</span> avgt <span class="pl-c1">10</span> <span class="pl-c1">31.111</span> ± <span class="pl-c1">0.720</span> us<span class="pl-k">/</span>op
[info] <span class="pl-en">RecursionBM</span>.cata2 <span class="pl-c1">1000</span> avgt <span class="pl-c1">10</span> <span class="pl-c1">16.067</span> ± <span class="pl-c1">0.187</span> us<span class="pl-k">/</span>op
[info] <span class="pl-en">RecursionBM</span>.cata <span class="pl-c1">10000</span> avgt <span class="pl-c1">10</span> <span class="pl-c1">326.443</span> ± <span class="pl-c1">9.054</span> us<span class="pl-k">/</span>op
[info] <span class="pl-en">RecursionBM</span>.cata2 <span class="pl-c1">10000</span> avgt <span class="pl-c1">10</span> <span class="pl-c1">165.470</span> ± <span class="pl-c1">1.617</span> us<span class="pl-k">/</span>op</pre></div>
<p>200%, it's twice as fast!
Not bad for a tiny bit of one-off, hidden boilerplate.</p>
<p><em>Side-note: I ran the benchmark on a i7-6700HQ and all results are under 1ms,
even at structure size of 1x10000 (length x depth), which means that either
implementation is going to be fine on a fast CPU in a non-high-throughput
solution. It'd be interesting to know what the measurements would be in prod,
on real GCP/AWS VMs; the savings of the optimisation would be more significant
because the VCPUs are slower.</em></p>
<h3><a href="#f-algebras" aria-hidden="true" class="anchor" id="user-content-f-algebras"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>F-Algebras</h3>
<p>This isn't necessary but I'm also going to add a type alias.</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">type</span> <span class="pl-en">FAlgebra</span>[<span class="pl-en">F</span>[_], <span class="pl-en">A</span>] <span class="pl-k">=</span> <span class="pl-en">F</span>[<span class="pl-en">A</span>] <span class="pl-k">=></span> <span class="pl-en">A</span></pre></div>
<p>and tweak the catamorphism definition to</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">def</span> <span class="pl-en">cata2</span>[<span class="pl-en">F</span>[_], <span class="pl-en">A</span>, <span class="pl-en">B</span>](<span class="pl-v">algebra</span>: <span class="pl-en">FAlgebra</span>[<span class="pl-en">F</span>, <span class="pl-en">A</span>])(<span class="pl-v">f</span>: <span class="pl-en">Fix</span>[<span class="pl-en">F</span>])(<span class="pl-k">implicit</span> <span class="pl-en">F</span><span class="pl-k">:</span> <span class="pl-en">Functor</span>[<span class="pl-en">F</span>])<span class="pl-k">:</span> <span class="pl-en">A</span> <span class="pl-k">=</span> {
<span class="pl-k">var</span> <span class="pl-en">self</span><span class="pl-k">:</span> <span class="pl-en">Fix</span>[<span class="pl-en">F</span>] <span class="pl-k">=></span> <span class="pl-en">A</span> <span class="pl-k">=</span> <span class="pl-c1">null</span>
<span class="pl-c1">self</span> <span class="pl-k">=</span> f <span class="pl-k">=></span> algebra(<span class="pl-en">F</span>.map(f.unfix)(<span class="pl-c1">self</span>))
<span class="pl-c1">self</span>(f)
}</pre></div>
<p>Type aliases don't exist at runtime, the compilation process dealiases them completely.
You can also use either definition interchangably.</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">def</span> <span class="pl-en">useAlias</span>[<span class="pl-en">F</span>[_], <span class="pl-en">A</span>](<span class="pl-v">f</span>: <span class="pl-en">F</span>[<span class="pl-en">A</span>] <span class="pl-k">=></span> <span class="pl-en">A</span>)<span class="pl-k">:</span> <span class="pl-en">FAlgebra</span>[<span class="pl-en">F</span>, <span class="pl-en">A</span>] <span class="pl-k">=</span>
f
<span class="pl-k">def</span> <span class="pl-en">removeAlias</span>[<span class="pl-en">F</span>[_], <span class="pl-en">A</span>](<span class="pl-v">f</span>: <span class="pl-en">FAlgebra</span>[<span class="pl-en">F</span>, <span class="pl-en">A</span>])<span class="pl-k">:</span> <span class="pl-en">F</span>[<span class="pl-en">A</span>] <span class="pl-k">=></span> <span class="pl-en">A</span> <span class="pl-k">=</span>
f</pre></div>
<p>There are two advantages to having and using a type alias.</p>
<ol>
<li>Readability. The <code>A</code> and the <code>F</code> are separated; the <code>A</code> doesn't repeat; it clarifies intent the more you get used to recursion schemes.</li>
<li>There are cases in which it helps type inference.</li>
</ol>
<h1><a href="#simple-examples" aria-hidden="true" class="anchor" id="user-content-simple-examples"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Simple Examples</h1>
<p>Let's start with some basic examples:
the usual <code>blah -> Int</code> stuff:</p>
<h3><a href="#list-sum" aria-hidden="true" class="anchor" id="user-content-list-sum"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>List Sum</h3>
<p>Let's sum a list:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">val</span> <span class="pl-en">listSum</span><span class="pl-k">:</span> <span class="pl-en">FAlgebra</span>[<span class="pl-en">IntListF</span>, <span class="pl-k">Int</span>] <span class="pl-k">=</span> {
<span class="pl-k">case</span> <span class="pl-en">IntListF</span>.<span class="pl-en">Cons</span>(h, t) <span class="pl-k">=></span> h <span class="pl-k">+</span> t
<span class="pl-k">case</span> <span class="pl-en">IntListF</span>.<span class="pl-c1">Nil</span> <span class="pl-k">=></span> <span class="pl-c1">0</span>
}</pre></div>
<p>How does it work?</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">val</span> <span class="pl-en">listSumVerbose</span><span class="pl-k">:</span> <span class="pl-en">IntListF</span>[<span class="pl-k">Int</span>] <span class="pl-k">=></span> <span class="pl-k">Int</span> <span class="pl-k">=</span> {
<span class="pl-k">case</span> <span class="pl-en">IntListF</span>.<span class="pl-en">Cons</span>(h, t) <span class="pl-k">=></span> h <span class="pl-k">+</span> t
<span class="pl-c"><span class="pl-c">//</span> | |</span>
<span class="pl-c"><span class="pl-c">//</span> Int by definition |</span>
<span class="pl-c"><span class="pl-c">//</span> Sum of tail (Int)</span>
<span class="pl-k">case</span> <span class="pl-en">IntListF</span>.<span class="pl-c1">Nil</span> <span class="pl-k">=></span> <span class="pl-c1">0</span>
}</pre></div>
<p>Notice this is an algebra, to actually use it in you need to call <code>cata</code>:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">def</span> <span class="pl-en">sumThisListPlease</span>(<span class="pl-v">list</span>: <span class="pl-en">IntList</span>)<span class="pl-k">:</span> <span class="pl-k">Int</span> <span class="pl-k">=</span>
cata(listSum)(list)</pre></div>
<p>For reasons that will become obvious later, when using this stuff in your own
project or libraries, the algebra itself is the unit that you'll be exposing most
often. Instead of creating functions that take fixed-point data (or codata)
structures and return a result, you create algebras and leave it to
users to call <code>cata</code> themselves. More on this later but the point is, from the
next example onwards, I'll show just the algebras.</p>
<h3><a href="#list-length" aria-hidden="true" class="anchor" id="user-content-list-length"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>List Length</h3>
<p>Counting elements in a list is similar:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">val</span> <span class="pl-en">listLength</span><span class="pl-k">:</span> <span class="pl-en">FAlgebra</span>[<span class="pl-en">IntListF</span>, <span class="pl-k">Int</span>] <span class="pl-k">=</span> {
<span class="pl-k">case</span> <span class="pl-en">IntListF</span>.<span class="pl-en">Cons</span>(_, t) <span class="pl-k">=></span> <span class="pl-c1">1</span> <span class="pl-k">+</span> t
<span class="pl-k">case</span> <span class="pl-en">IntListF</span>.<span class="pl-c1">Nil</span> <span class="pl-k">=></span> <span class="pl-c1">0</span>
}</pre></div>
<p>How does it work?</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">val</span> <span class="pl-en">listLengthVerbose</span><span class="pl-k">:</span> <span class="pl-en">IntListF</span>[<span class="pl-k">Int</span>] <span class="pl-k">=></span> <span class="pl-k">Int</span> <span class="pl-k">=</span> {
<span class="pl-k">case</span> <span class="pl-en">IntListF</span>.<span class="pl-en">Cons</span>(_, t) <span class="pl-k">=></span> <span class="pl-c1">1</span> <span class="pl-k">+</span> t
<span class="pl-c"><span class="pl-c">//</span> | |</span>
<span class="pl-c"><span class="pl-c">//</span> | Add 1 for this element</span>
<span class="pl-c"><span class="pl-c">//</span> Length of tail (Int)</span>
<span class="pl-k">case</span> <span class="pl-en">IntListF</span>.<span class="pl-c1">Nil</span> <span class="pl-k">=></span> <span class="pl-c1">0</span>
}</pre></div>
<h3><a href="#binarytree-algebras" aria-hidden="true" class="anchor" id="user-content-binarytree-algebras"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>BinaryTree Algebras</h3>
<p>Here's a few algebras for <code>BinaryTree</code>:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">val</span> <span class="pl-en">binaryTreeNodeCount</span><span class="pl-k">:</span> <span class="pl-en">FAlgebra</span>[<span class="pl-en">BinaryTreeF</span>[<span class="pl-en">Any</span>, <span class="pl-k">?</span>], <span class="pl-k">Int</span>] <span class="pl-k">=</span> {
<span class="pl-k">case</span> <span class="pl-en">BinaryTreeF</span>.<span class="pl-en">Node</span>(left, _, right) <span class="pl-k">=></span> left <span class="pl-k">+</span> <span class="pl-c1">1</span> <span class="pl-k">+</span> right
<span class="pl-k">case</span> <span class="pl-en">BinaryTreeF</span>.<span class="pl-en">Leaf</span> <span class="pl-k">=></span> <span class="pl-c1">0</span>
}
<span class="pl-k">val</span> <span class="pl-en">binaryTreeMaxDepth</span><span class="pl-k">:</span> <span class="pl-en">FAlgebra</span>[<span class="pl-en">BinaryTreeF</span>[<span class="pl-en">Any</span>, <span class="pl-k">?</span>], <span class="pl-k">Int</span>] <span class="pl-k">=</span> {
<span class="pl-k">case</span> <span class="pl-en">BinaryTreeF</span>.<span class="pl-en">Node</span>(left, _, right) <span class="pl-k">=></span> left.max(right) <span class="pl-k">+</span> <span class="pl-c1">1</span>
<span class="pl-k">case</span> <span class="pl-en">BinaryTreeF</span>.<span class="pl-en">Leaf</span> <span class="pl-k">=></span> <span class="pl-c1">0</span>
}
<span class="pl-k">def</span> <span class="pl-en">binaryTreeSum</span>[<span class="pl-en">N</span>](<span class="pl-k">implicit</span> <span class="pl-en">N</span><span class="pl-k">:</span> <span class="pl-en">Numeric</span>[<span class="pl-en">N</span>])<span class="pl-k">:</span> <span class="pl-en">FAlgebra</span>[<span class="pl-en">BinaryTreeF</span>[<span class="pl-en">N</span>, <span class="pl-k">?</span>], <span class="pl-en">N</span>] <span class="pl-k">=</span> {
<span class="pl-k">case</span> <span class="pl-en">BinaryTreeF</span>.<span class="pl-en">Node</span>(left, n, right) <span class="pl-k">=></span> <span class="pl-en">N</span>.plus(left, <span class="pl-en">N</span>.plus(n, right))
<span class="pl-k">case</span> <span class="pl-en">BinaryTreeF</span>.<span class="pl-en">Leaf</span> <span class="pl-k">=></span> <span class="pl-en">N</span>.zero
}</pre></div>
<p>Pretty straight-forward. Each recursive slot (<code>left</code> & <code>right</code>) already has the
computed value for that subtree.</p>
<h3><a href="#json" aria-hidden="true" class="anchor" id="user-content-json"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>JSON</h3>
<p>Let's take a JSON value in our JSON ADT, and turn it into a JSON string that we can
send out the door.</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">val</span> <span class="pl-en">jsonToString</span><span class="pl-k">:</span> <span class="pl-en">FAlgebra</span>[<span class="pl-en">JsonF</span>, <span class="pl-k">String</span>] <span class="pl-k">=</span> {
<span class="pl-k">case</span> <span class="pl-en">JsonF</span>.<span class="pl-en">Null</span> <span class="pl-k">=></span> <span class="pl-s"><span class="pl-pds">"</span>null<span class="pl-pds">"</span></span>
<span class="pl-k">case</span> <span class="pl-en">JsonF</span>.<span class="pl-en">Bool</span>(b) <span class="pl-k">=></span> b.toString
<span class="pl-k">case</span> <span class="pl-en">JsonF</span>.<span class="pl-en">Num</span>(n) <span class="pl-k">=></span> n.toString
<span class="pl-k">case</span> <span class="pl-en">JsonF</span>.<span class="pl-en">Str</span>(s) <span class="pl-k">=></span> escapeString(s)
<span class="pl-k">case</span> <span class="pl-en">JsonF</span>.<span class="pl-en">Arr</span>(values) <span class="pl-k">=></span> values.mkString(<span class="pl-s"><span class="pl-pds">"</span>[<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>,<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>]<span class="pl-pds">"</span></span>)
<span class="pl-k">case</span> <span class="pl-en">JsonF</span>.<span class="pl-en">Obj</span>(fields) <span class="pl-k">=></span> fields.iterator.map { <span class="pl-k">case</span> (k, v) <span class="pl-k">=></span> k <span class="pl-k">+</span> <span class="pl-s"><span class="pl-pds">"</span>:<span class="pl-pds">"</span></span> <span class="pl-k">+</span> v }.mkString(<span class="pl-s"><span class="pl-pds">"</span>{<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>,<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>}<span class="pl-pds">"</span></span>)
}</pre></div>
<p>Is that easy or what? The array values and object fields are already all strings,
we just mindlessly combine them using array/object notation.</p>
<p>What if, instead of the slower String concatenation, we wanted to use
<code>StringBuilder</code>? Don't let the mutability discourage you; it's the same concept,
we'll just replace <code>String</code> in the algebra type signature with
<code>StringBuilder => Unit</code>. <em>Executing</em> a <code>StringBuilder => Unit</code> is mutable but
the function itself is immutable, referentially transparent and pure.
Descriptions of side-effects are safe.</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">val</span> <span class="pl-en">jsonToStringSB</span><span class="pl-k">:</span> <span class="pl-en">FAlgebra</span>[<span class="pl-en">JsonF</span>, <span class="pl-en">StringBuilder</span> <span class="pl-k">=></span> <span class="pl-k">Unit</span>] <span class="pl-k">=</span> {
<span class="pl-k">case</span> <span class="pl-en">JsonF</span>.<span class="pl-en">Null</span> <span class="pl-k">=></span> _ append <span class="pl-s"><span class="pl-pds">"</span>null<span class="pl-pds">"</span></span>
<span class="pl-k">case</span> <span class="pl-en">JsonF</span>.<span class="pl-en">Bool</span>(b) <span class="pl-k">=></span> _ append b.toString
<span class="pl-k">case</span> <span class="pl-en">JsonF</span>.<span class="pl-en">Num</span>(n) <span class="pl-k">=></span> _ append n.toString
<span class="pl-k">case</span> <span class="pl-en">JsonF</span>.<span class="pl-en">Str</span>(s) <span class="pl-k">=></span> _ append escapeString(s)
<span class="pl-k">case</span> <span class="pl-en">JsonF</span>.<span class="pl-en">Arr</span>(values) <span class="pl-k">=></span> sb <span class="pl-k">=></span> {
sb append <span class="pl-c1">'['</span>
<span class="pl-k">for</span> (v <span class="pl-k"><</span><span class="pl-k">-</span> values) v(sb)
sb append <span class="pl-c1">']'</span>
}
<span class="pl-k">case</span> <span class="pl-en">JsonF</span>.<span class="pl-en">Obj</span>(fields) <span class="pl-k">=></span> sb <span class="pl-k">=></span> {
sb append <span class="pl-c1">'{'</span>
<span class="pl-k">for</span> ((k, v) <span class="pl-k"><</span><span class="pl-k">-</span> fields) {
sb append k
sb append <span class="pl-c1">':'</span>
v(sb)
}
sb append <span class="pl-c1">'}'</span>
}
}</pre></div>
<p>To be clear, usage would look like this:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">def</span> <span class="pl-en">jsonToStringBuilderUsage</span>(<span class="pl-v">json</span>: <span class="pl-en">Json</span>)<span class="pl-k">:</span> <span class="pl-k">String</span> <span class="pl-k">=</span> {
<span class="pl-k">val</span> <span class="pl-en">sbToUnit</span> <span class="pl-k">=</span> cata(jsonToStringSB)(json)
<span class="pl-k">val</span> <span class="pl-en">sb</span> <span class="pl-k">=</span> <span class="pl-k">new</span> <span class="pl-en">StringBuilder</span>
sbToUnit(sb)
sb.toString()
}</pre></div>
<h1><a href="#a-file-system" aria-hidden="true" class="anchor" id="user-content-a-file-system"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>A File System</h1>
<p>Let's look at a more interesting example: a file system.</p>
<p>We'll start with a typical representation with hard-coded recursion.</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">Entry</span>
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Dir</span>(<span class="pl-v">files</span>: <span class="pl-en">Map</span>[<span class="pl-k">String</span>, <span class="pl-en">Entry</span>]) <span class="pl-k">extends</span> <span class="pl-e">Entry</span>
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">File</span>(<span class="pl-v">size</span>: <span class="pl-k">Long</span>) <span class="pl-k">extends</span> <span class="pl-e">Entry</span></pre></div>
<p>This is an example inhabitant.</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-c"><span class="pl-c">//</span> Example of 3 files:</span>
<span class="pl-c"><span class="pl-c">//</span> 1. /usr/bin/find</span>
<span class="pl-c"><span class="pl-c">//</span> 2. /usr/bin/ls</span>
<span class="pl-c"><span class="pl-c">//</span> 3. /tmp/example.tmp</span>
<span class="pl-k">val</span> <span class="pl-en">example</span> <span class="pl-k">=</span>
<span class="pl-en">Dir</span>(<span class="pl-en">Map</span>(
<span class="pl-s"><span class="pl-pds">"</span>usr<span class="pl-pds">"</span></span> <span class="pl-k">-</span><span class="pl-k">></span> <span class="pl-en">Dir</span>(<span class="pl-en">Map</span>(
<span class="pl-s"><span class="pl-pds">"</span>bin<span class="pl-pds">"</span></span> <span class="pl-k">-</span><span class="pl-k">></span> <span class="pl-en">Dir</span>(<span class="pl-en">Map</span>(
<span class="pl-s"><span class="pl-pds">"</span>find<span class="pl-pds">"</span></span> <span class="pl-k">-</span><span class="pl-k">></span> <span class="pl-en">File</span>(<span class="pl-c1">197360</span>),
<span class="pl-s"><span class="pl-pds">"</span>ls<span class="pl-pds">"</span></span> <span class="pl-k">-</span><span class="pl-k">></span> <span class="pl-en">File</span>(<span class="pl-c1">133688</span>))))),
<span class="pl-s"><span class="pl-pds">"</span>tmp<span class="pl-pds">"</span></span> <span class="pl-k">-</span><span class="pl-k">></span> <span class="pl-en">Dir</span>(<span class="pl-en">Map</span>(
<span class="pl-s"><span class="pl-pds">"</span>example.tmp<span class="pl-pds">"</span></span> <span class="pl-k">-</span><span class="pl-k">></span> <span class="pl-en">File</span>(<span class="pl-c1">12</span>)))))</pre></div>
<p>Now let's create an API:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">def</span> <span class="pl-en">totalFileSize</span>(<span class="pl-v">e</span>: <span class="pl-en">Entry</span>)<span class="pl-k">:</span> <span class="pl-k">Long</span> <span class="pl-k">=</span> e <span class="pl-k">match</span> {
<span class="pl-k">case</span> <span class="pl-en">File</span>(s) <span class="pl-k">=></span> s
<span class="pl-k">case</span> <span class="pl-en">Dir</span>(fs) <span class="pl-k">=></span> fs.values.foldLeft(<span class="pl-c1">0L</span>)(_ <span class="pl-k">+</span> totalFileSize(_))
}
<span class="pl-k">def</span> <span class="pl-en">countFiles</span>(<span class="pl-v">e</span>: <span class="pl-en">Entry</span>)<span class="pl-k">:</span> <span class="pl-k">Int</span> <span class="pl-k">=</span> e <span class="pl-k">match</span> {
<span class="pl-k">case</span> <span class="pl-en">File</span>(_) <span class="pl-k">=></span> <span class="pl-c1">1</span>
<span class="pl-k">case</span> <span class="pl-en">Dir</span>(fs) <span class="pl-k">=></span> fs.values.foldLeft(<span class="pl-c1">0</span>)(_ <span class="pl-k">+</span> countFiles(_))
}
<span class="pl-k">def</span> <span class="pl-en">countDirs</span>(<span class="pl-v">e</span>: <span class="pl-en">Entry</span>)<span class="pl-k">:</span> <span class="pl-k">Int</span> <span class="pl-k">=</span> e <span class="pl-k">match</span> {
<span class="pl-k">case</span> <span class="pl-en">File</span>(_) <span class="pl-k">=></span> <span class="pl-c1">0</span>
<span class="pl-k">case</span> <span class="pl-en">Dir</span>(fs) <span class="pl-k">=></span> fs.values.foldLeft(<span class="pl-c1">1</span>)(_ <span class="pl-k">+</span> countDirs(_))
}</pre></div>
<p>Looks great! SHIP IT!
Ok so now people are using our super-awesome file system and associated API.
One day a user wants to collect a bunch of stats and writes the following code:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Stats</span>(<span class="pl-v">totalSize</span>: <span class="pl-k">Long</span>, <span class="pl-v">files</span>: <span class="pl-k">Int</span>, <span class="pl-v">dirs</span>: <span class="pl-k">Int</span>)
<span class="pl-k">def</span> <span class="pl-en">stats</span>(<span class="pl-v">e</span>: <span class="pl-en">Entry</span>)<span class="pl-k">:</span> <span class="pl-en">Stats</span> <span class="pl-k">=</span>
<span class="pl-en">Stats</span>(totalFileSize(e), countFiles(e), countDirs(e))</pre></div>
<p>The user then complains that their <code>stats</code> method takes 3 times as long as other
operations.
This is because each stat is produced by traversing the entire file system.
3 stats = 3 traversals.
Now obviously, with the pure definitions given above, the extra time
is going to be negligible but imagine it's a real file system here, maybe even one
distributed over the network, all that drive/network/hardware cost to traverse
the file system is likely to be very noticable and very significant when it's
repeated 3 times.</p>
<p>What can the user do? Well... nothing. The only recourse they have is to raise an
issue and complain. They have no control or power in this situation.
They're at the mercy of the decisions made by the library authors.</p>
<p>After a few years of complaints, the authors of the super-awesome file system
library do a big rewrite and create a new API
that looks a little something like this...</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Stats</span>(<span class="pl-v">totalSize</span>: <span class="pl-k">Long</span>, <span class="pl-v">files</span>: <span class="pl-k">Int</span>, <span class="pl-v">dirs</span>: <span class="pl-k">Int</span>)
<span class="pl-k">def</span> <span class="pl-en">stats</span>(<span class="pl-v">e</span>: <span class="pl-en">Entry</span>)<span class="pl-k">:</span> <span class="pl-en">Stats</span> <span class="pl-k">=</span> e <span class="pl-k">match</span> {
<span class="pl-k">case</span> <span class="pl-en">File</span>(fileSize) <span class="pl-k">=></span>
<span class="pl-en">Stats</span>(fileSize, <span class="pl-c1">1</span>, <span class="pl-c1">0</span>)
<span class="pl-k">case</span> <span class="pl-en">Dir</span>(fs) <span class="pl-k">=></span>
fs.values.foldLeft(<span class="pl-en">Stats</span>(<span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>)) { (statsAcc, entry) <span class="pl-k">=></span>
<span class="pl-k">val</span> <span class="pl-en">b</span> <span class="pl-k">=</span> stats(entry)
<span class="pl-en">Stats</span>(
statsAcc.totalSize <span class="pl-k">+</span> b.totalSize,
statsAcc.files <span class="pl-k">+</span> b.files,
statsAcc.dirs <span class="pl-k">+</span> b.dirs)
}
}
<span class="pl-k">def</span> <span class="pl-en">totalFileSize</span>(<span class="pl-v">e</span>: <span class="pl-en">Entry</span>)<span class="pl-k">:</span> <span class="pl-k">Long</span> <span class="pl-k">=</span>
stats(e).totalSize
<span class="pl-k">def</span> <span class="pl-en">countFiles</span>(<span class="pl-v">e</span>: <span class="pl-en">Entry</span>)<span class="pl-k">:</span> <span class="pl-k">Int</span> <span class="pl-k">=</span>
stats(e).files
<span class="pl-k">def</span> <span class="pl-en">countDirs</span>(<span class="pl-v">e</span>: <span class="pl-en">Entry</span>)<span class="pl-k">:</span> <span class="pl-k">Int</span> <span class="pl-k">=</span>
stats(e).dirs</pre></div>
<p>KICK-ASS!
Now all stats are gathered in a single traversal; issue: resolved.
Except there are two problems now where there once was one.
First, over the next few versions of the library more and more stats
(permissions, ownership, etc.) get added, and the <code>stats</code> function gets bigger,
more complex and more wild.
Second, now new complaints start coming in, complaints like
<em>"totalFileSize() significantly slower since new update"</em>,
<em>"dirCount() 4x slower than in v1.8.3"</em>. What's going on?
Well, now in every traversal we're spending more time fetching more data that
we don't need and end up throwing away. If a user only wants a directory count,
the library spends oodles of time looking up attributes, group names, etc. only
to discard them all at the end.</p>
<p>What can the user do? Well... again: nothing. The only recourse they have is to raise an
issue and complain. They have no control or power in this situation.
They're yet again at the mercy of the decisions made by the library authors.</p>
<h3><a href="#fixing-our-filesystem" aria-hidden="true" class="anchor" id="user-content-fixing-our-filesystem"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Fixing our FileSystem</h3>
<p>Start with the usual changes: type param, <code>Functor</code>, <code>Fix</code>:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">EntryF</span>[<span class="pl-k">+</span><span class="pl-en">F</span>]
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Dir</span>[<span class="pl-en">F</span>](<span class="pl-v">files</span>: <span class="pl-en">Map</span>[<span class="pl-k">String</span>, <span class="pl-en">F</span>]) <span class="pl-k">extends</span> <span class="pl-e">EntryF</span>[<span class="pl-en">F</span>]
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">File</span>(<span class="pl-v">size</span>: <span class="pl-k">Long</span>) <span class="pl-k">extends</span> <span class="pl-e">EntryF</span>[<span class="pl-en">Nothing</span>]
<span class="pl-k">object</span> <span class="pl-en">EntryF</span> {
<span class="pl-k">implicit</span> <span class="pl-k">val</span> <span class="pl-en">functor</span><span class="pl-k">:</span> <span class="pl-en">Functor</span>[<span class="pl-en">EntryF</span>] <span class="pl-k">=</span> <span class="pl-k">new</span> <span class="pl-en">Functor</span>[<span class="pl-en">EntryF</span>] {
<span class="pl-k">override</span> <span class="pl-k">def</span> <span class="pl-en">map</span>[<span class="pl-en">A</span>, <span class="pl-en">B</span>](<span class="pl-v">fa</span>: <span class="pl-en">EntryF</span>[<span class="pl-en">A</span>])(<span class="pl-v">f</span>: <span class="pl-en">A</span> <span class="pl-k">=></span> <span class="pl-en">B</span>)<span class="pl-k">:</span> <span class="pl-en">EntryF</span>[<span class="pl-en">B</span>] <span class="pl-k">=</span> fa <span class="pl-k">match</span> {
<span class="pl-k">case</span> <span class="pl-v">f</span>: <span class="pl-en">File</span> <span class="pl-k">=></span> f
<span class="pl-k">case</span> <span class="pl-en">Dir</span>(fs) <span class="pl-k">=></span> <span class="pl-en">Dir</span>(fs.map { <span class="pl-k">case</span> (k, v) <span class="pl-k">=></span> (k, f(v)) })
}
}
}
<span class="pl-k">type</span> <span class="pl-en">Entry</span> <span class="pl-k">=</span> <span class="pl-en">Fix</span>[<span class="pl-en">EntryF</span>]</pre></div>
<p>Now let's add a little DSL and re-create our sample value:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">object</span> <span class="pl-en">Entry</span> {
<span class="pl-k">def</span> <span class="pl-en">apply</span>(<span class="pl-v">f</span>: <span class="pl-en">EntryF</span>[<span class="pl-en">Entry</span>])<span class="pl-k">:</span> <span class="pl-en">Entry</span> <span class="pl-k">=</span> <span class="pl-en">Fix</span>(f)
<span class="pl-k">def</span> <span class="pl-en">file</span>(<span class="pl-v">s</span>: <span class="pl-k">Long</span>)<span class="pl-k">:</span> <span class="pl-en">Entry</span> <span class="pl-k">=</span> apply(<span class="pl-en">File</span>(s))
<span class="pl-k">def</span> <span class="pl-en">dir</span>(<span class="pl-v">es</span>: (<span class="pl-k">String</span>, <span class="pl-en">Entry</span>)<span class="pl-k">*</span>)<span class="pl-k">:</span> <span class="pl-en">Entry</span> <span class="pl-k">=</span> apply(<span class="pl-en">Dir</span>(es.toMap))
}
<span class="pl-c"><span class="pl-c">//</span> Example of 3 files:</span>
<span class="pl-c"><span class="pl-c">//</span> 1. /usr/bin/find</span>
<span class="pl-c"><span class="pl-c">//</span> 2. /usr/bin/ls</span>
<span class="pl-c"><span class="pl-c">//</span> 3. /tmp/example.tmp</span>
<span class="pl-k">val</span> <span class="pl-en">example</span> <span class="pl-k">=</span>
<span class="pl-en">Entry</span>.dir(
<span class="pl-s"><span class="pl-pds">"</span>usr<span class="pl-pds">"</span></span> <span class="pl-k">-</span><span class="pl-k">></span> <span class="pl-en">Entry</span>.dir(
<span class="pl-s"><span class="pl-pds">"</span>bin<span class="pl-pds">"</span></span> <span class="pl-k">-</span><span class="pl-k">></span> <span class="pl-en">Entry</span>.dir(
<span class="pl-s"><span class="pl-pds">"</span>find<span class="pl-pds">"</span></span> <span class="pl-k">-</span><span class="pl-k">></span> <span class="pl-en">Entry</span>.file(<span class="pl-c1">197360</span>),
<span class="pl-s"><span class="pl-pds">"</span>ls<span class="pl-pds">"</span></span> <span class="pl-k">-</span><span class="pl-k">></span> <span class="pl-en">Entry</span>.file(<span class="pl-c1">133688</span>))),
<span class="pl-s"><span class="pl-pds">"</span>tmp<span class="pl-pds">"</span></span> <span class="pl-k">-</span><span class="pl-k">></span> <span class="pl-en">Entry</span>.dir(
<span class="pl-s"><span class="pl-pds">"</span>example.tmp<span class="pl-pds">"</span></span> <span class="pl-k">-</span><span class="pl-k">></span> <span class="pl-en">Entry</span>.file(<span class="pl-c1">12</span>)))</pre></div>
<p>Next up are the queries.
Small, reusable, independent functions are good practice for good reason;
let's recreate what we had in the initial version:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">val</span> <span class="pl-en">totalFileSize</span><span class="pl-k">:</span> <span class="pl-en">FAlgebra</span>[<span class="pl-en">EntryF</span>, <span class="pl-k">Long</span>] <span class="pl-k">=</span> {
<span class="pl-k">case</span> <span class="pl-en">File</span>(s) <span class="pl-k">=></span> s
<span class="pl-k">case</span> <span class="pl-en">Dir</span>(fs) <span class="pl-k">=></span> fs.values.sum
}
<span class="pl-k">val</span> <span class="pl-en">countFiles</span><span class="pl-k">:</span> <span class="pl-en">FAlgebra</span>[<span class="pl-en">EntryF</span>, <span class="pl-k">Int</span>] <span class="pl-k">=</span> {
<span class="pl-k">case</span> <span class="pl-en">File</span>(_) <span class="pl-k">=></span> <span class="pl-c1">1</span>
<span class="pl-k">case</span> <span class="pl-en">Dir</span>(fs) <span class="pl-k">=></span> fs.values.sum
}
<span class="pl-k">val</span> <span class="pl-en">countDirs</span><span class="pl-k">:</span> <span class="pl-en">FAlgebra</span>[<span class="pl-en">EntryF</span>, <span class="pl-k">Int</span>] <span class="pl-k">=</span> {
<span class="pl-k">case</span> <span class="pl-en">File</span>(_) <span class="pl-k">=></span> <span class="pl-c1">0</span>
<span class="pl-k">case</span> <span class="pl-en">Dir</span>(fs) <span class="pl-k">=></span> fs.values.sum <span class="pl-k">+</span> <span class="pl-c1">1</span>
}</pre></div>
<p>If you compare this to the first attempt, it's just as simple from the outside,
and even simpler in its internal implementation!</p>
<p>Now what happens when a user wants to get all three stats?
F-Algebras compose. Here's a simple, reusable snippet to combine two F-algebras
that share the same <code>F</code>:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">def</span> <span class="pl-en">algebraZip</span>[<span class="pl-en">F</span>[_], <span class="pl-en">A</span>, <span class="pl-en">B</span>](<span class="pl-v">fa</span>: <span class="pl-en">FAlgebra</span>[<span class="pl-en">F</span>, <span class="pl-en">A</span>],
<span class="pl-v">fb</span>: <span class="pl-en">FAlgebra</span>[<span class="pl-en">F</span>, <span class="pl-en">B</span>])
(<span class="pl-k">implicit</span> <span class="pl-en">F</span><span class="pl-k">:</span> <span class="pl-en">Functor</span>[<span class="pl-en">F</span>])<span class="pl-k">:</span> <span class="pl-en">FAlgebra</span>[<span class="pl-en">F</span>, (<span class="pl-en">A</span>, <span class="pl-en">B</span>)] <span class="pl-k">=</span>
fab <span class="pl-k">=></span> {
<span class="pl-k">val</span> <span class="pl-en">a</span> <span class="pl-k">=</span> fa(fab.map(_._1))
<span class="pl-k">val</span> <span class="pl-en">b</span> <span class="pl-k">=</span> fb(fab.map(_._2))
(a, b)
}</pre></div>
<p>This is all the user needs to create their own stats gathering algebra:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">val</span> <span class="pl-en">statsAlg</span><span class="pl-k">:</span> <span class="pl-en">FAlgebra</span>[<span class="pl-en">EntryF</span>, (<span class="pl-k">Long</span>, (<span class="pl-k">Int</span>, <span class="pl-k">Int</span>))] <span class="pl-k">=</span>
algebraZip(totalFileSize, algebraZip(countFiles, countDirs))</pre></div>
<p>The <code>(Long, (Int, Int))</code> is a little ugly, let's clean it up a bit:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Stats</span>(<span class="pl-v">totalSize</span>: <span class="pl-k">Long</span>, <span class="pl-v">files</span>: <span class="pl-k">Int</span>, <span class="pl-v">dirs</span>: <span class="pl-k">Int</span>)
<span class="pl-k">def</span> <span class="pl-en">stats</span>(<span class="pl-v">e</span>: <span class="pl-en">Entry</span>)<span class="pl-k">:</span> <span class="pl-en">Stats</span> <span class="pl-k">=</span> {
<span class="pl-k">val</span> (totalSize, (files, dirs)) <span class="pl-k">=</span> cata(statsAlg)(e)
<span class="pl-en">Stats</span>(totalSize, files, dirs)
}</pre></div>
<p>Hoorah. Without any dependencies on the library authors, our user was able to
choose the 3 stats they were interested in, and retrieve them all in a single
file system traversal.
Whilst not parallelism (wait for the next episode in this series),
they've created their own concurrency!</p>
<p>Now contrast this with the previous incarnation.
This time around, the user has the control, the user has the power.
Library authors don't need to decide tradeoffs for everyone.</p>
<p>These are the advantages of providing algebras instead of higher-purpose functions.
F-algebras (like <code>val statsAlg</code>) allow you to accrue power;
that power is spent when you apply it (like <code>def stats</code>).</p>
<p>Personally speaking, when using all this stuff in my own work projects,
I often find it nice to provide two sets of interfaces:</p>
<ol>
<li>A low-level ecosystem of algebras, raw data types, logic, etc.
Usually an entire package.</li>
<li>A high-level DSL that very cleanly exposes common high-level functions and
hides all the algebra composition, calls to <code>cata</code> and other morphisms, etc.
Usually a single <code>object</code>.</li>
</ol>
<p>This approach allows all devs to get the most common types of work done without
any knowledge of recursion schemes or theory.
They just see a small, high-level DSL that's easy to skim, understand and use.
It also serves as a good means of teaching recursion schemes because
devs unfamiliar with it can then click through into the lower-level code and see
real-world examples in a domain they're familiar with.
This makes for a nice experience on teams with differing skill levels,
both in terms of code quality and a path to gradually up-skill the team.</p>
<h1><a href="#to-be-continued" aria-hidden="true" class="anchor" id="user-content-to-be-continued"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>To be continued...</h1>
<p>That's plenty for now.</p>
<p>If you like what I do and you'd like to support me, this series or any of my other projects,
<a href="https://www.patreon.com/japgolly" rel="nofollow">become a patron!</a>
It will make a big difference and help further more content.
You can also <a href="https://twitter.com/japgolly" rel="nofollow">follow me on Twitter</a>
for updates about the next post the series.</p>
<p>All source code <a href="https://github.com/japgolly/learning/tree/public/recursion-blog/example/src/main/scala/japgolly/blog/recursion">available here</a>.</p>
<p>Looking for a Scala recursion scheme library?
There's a list <a href="https://github.com/japgolly/learning/blob/public/recursion-blog/FAQ.md">in the FAQ</a>.</p>
</article>
</div>
</div>
</div>
</div>
</div>
</div>
David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com5tag:blogger.com,1999:blog-5837865295676240265.post-7608939535391592112017-11-20T18:15:00.001+11:002017-12-13T11:30:32.180+11:00Practical Awesome Recursion - Ch 01: Fixpoints<style>
.goost{font-size:16px;color:#333;text-align:left;/*!
* GitHub Light v0.4.1
* Copyright (c) 2012 - 2017 GitHub, Inc.
* Licensed under MIT (https://github.com/primer/github-syntax-theme-generator/blob/master/LICENSE)
*/direction:ltr}.goost .markdown-body{font-family:-apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol";font-size:16px;line-height:1.5;word-wrap:break-word}.goost .markdown-body::before{display:table;content:""}.goost .markdown-body::after{display:table;clear:both;content:""}.goost .markdown-body>*:first-child{margin-top:0 !important}.goost .markdown-body>*:last-child{margin-bottom:0 !important}.goost .markdown-body a:not([href]){color:inherit;text-decoration:none}.goost .markdown-body .absent{color:#cb2431}.goost .markdown-body .anchor{float:left;padding-right:4px;margin-left:-20px;line-height:1}.goost .markdown-body .anchor:focus{outline:none}.goost .markdown-body p,.goost .markdown-body blockquote,.goost .markdown-body ul,.goost .markdown-body ol,.goost .markdown-body dl,.goost .markdown-body table,.goost .markdown-body pre{margin-top:0;margin-bottom:16px}.goost .markdown-body hr{height:0.25em;padding:0;margin:24px 0;background-color:#e1e4e8;border:0}.goost .markdown-body blockquote{padding:0 1em;color:#6a737d;border-left:0.25em solid #dfe2e5}.goost .markdown-body blockquote>:first-child{margin-top:0}.goost .markdown-body blockquote>:last-child{margin-bottom:0}.goost .markdown-body kbd{display:inline-block;padding:3px 5px;font-size:11px;line-height:10px;color:#444d56;vertical-align:middle;background-color:#fafbfc;border:solid 1px #c6cbd1;border-bottom-color:#959da5;border-radius:3px;box-shadow:inset 0 -1px 0 #959da5}.goost .markdown-body h1,.goost .markdown-body h2,.goost .markdown-body h3,.goost .markdown-body h4,.goost .markdown-body h5,.goost .markdown-body h6{margin-top:24px;margin-bottom:16px;font-weight:600;line-height:1.25}.goost .markdown-body h1 .octicon-link,.goost .markdown-body h2 .octicon-link,.goost .markdown-body h3 .octicon-link,.goost .markdown-body h4 .octicon-link,.goost .markdown-body h5 .octicon-link,.goost .markdown-body h6 .octicon-link{color:#1b1f23;vertical-align:middle;visibility:hidden}.goost .markdown-body h1:hover .anchor,.goost .markdown-body h2:hover .anchor,.goost .markdown-body h3:hover .anchor,.goost .markdown-body h4:hover .anchor,.goost .markdown-body h5:hover .anchor,.goost .markdown-body h6:hover .anchor{text-decoration:none}.goost .markdown-body h1:hover .anchor .octicon-link,.goost .markdown-body h2:hover .anchor .octicon-link,.goost .markdown-body h3:hover .anchor .octicon-link,.goost .markdown-body h4:hover .anchor .octicon-link,.goost .markdown-body h5:hover .anchor .octicon-link,.goost .markdown-body h6:hover .anchor .octicon-link{visibility:visible}.goost .markdown-body h1 tt,.goost .markdown-body h1 code,.goost .markdown-body h2 tt,.goost .markdown-body h2 code,.goost .markdown-body h3 tt,.goost .markdown-body h3 code,.goost .markdown-body h4 tt,.goost .markdown-body h4 code,.goost .markdown-body h5 tt,.goost .markdown-body h5 code,.goost .markdown-body h6 tt,.goost .markdown-body h6 code{font-size:inherit}.goost .markdown-body h1{padding-bottom:0.3em;font-size:2em;border-bottom:1px solid #eaecef}.goost .markdown-body h2{padding-bottom:0.3em;font-size:1.5em;border-bottom:1px solid #eaecef}.goost .markdown-body h3{font-size:1.25em}.goost .markdown-body h4{font-size:1em}.goost .markdown-body h5{font-size:0.875em}.goost .markdown-body h6{font-size:0.85em;color:#6a737d}.goost .markdown-body ul,.goost .markdown-body ol{padding-left:2em}.goost .markdown-body ul.no-list,.goost .markdown-body ol.no-list{padding:0;list-style-type:none}.goost .markdown-body ul ul,.goost .markdown-body ul ol,.goost .markdown-body ol ol,.goost .markdown-body ol ul{margin-top:0;margin-bottom:0}.goost .markdown-body li{word-wrap:break-all}.goost .markdown-body li>p{margin-top:16px}.goost .markdown-body li+li{margin-top:0.25em}.goost .markdown-body dl{padding:0}.goost .markdown-body dl dt{padding:0;margin-top:16px;font-size:1em;font-style:italic;font-weight:600}.goost .markdown-body dl dd{padding:0 16px;margin-bottom:16px}.goost .markdown-body table{display:block;width:100%;overflow:auto}.goost .markdown-body table th{font-weight:600}.goost .markdown-body table th,.goost .markdown-body table td{padding:6px 13px;border:1px solid #dfe2e5}.goost .markdown-body table tr{background-color:#fff;border-top:1px solid #c6cbd1}.goost .markdown-body table tr:nth-child(2n){background-color:#f6f8fa}.goost .markdown-body table img{background-color:transparent}.goost .markdown-body img{max-width:100%;box-sizing:content-box;background-color:#fff}.goost .markdown-body img[align=right]{padding-left:20px}.goost .markdown-body img[align=left]{padding-right:20px}.goost .markdown-body .emoji{max-width:none;vertical-align:text-top;background-color:transparent}.goost .markdown-body span.frame{display:block;overflow:hidden}.goost .markdown-body span.frame>span{display:block;float:left;width:auto;padding:7px;margin:13px 0 0;overflow:hidden;border:1px solid #dfe2e5}.goost .markdown-body span.frame span img{display:block;float:left}.goost .markdown-body span.frame span span{display:block;padding:5px 0 0;clear:both;color:#24292e}.goost .markdown-body span.align-center{display:block;overflow:hidden;clear:both}.goost .markdown-body span.align-center>span{display:block;margin:13px auto 0;overflow:hidden;text-align:center}.goost .markdown-body span.align-center span img{margin:0 auto;text-align:center}.goost .markdown-body span.align-right{display:block;overflow:hidden;clear:both}.goost .markdown-body span.align-right>span{display:block;margin:13px 0 0;overflow:hidden;text-align:right}.goost .markdown-body span.align-right span img{margin:0;text-align:right}.goost .markdown-body span.float-left{display:block;float:left;margin-right:13px;overflow:hidden}.goost .markdown-body span.float-left span{margin:13px 0 0}.goost .markdown-body span.float-right{display:block;float:right;margin-left:13px;overflow:hidden}.goost .markdown-body span.float-right>span{display:block;margin:13px auto 0;overflow:hidden;text-align:right}.goost .markdown-body code,.goost .markdown-body tt{padding:0.2em 0.4em;margin:0;font-size:85%;background-color:rgba(27,31,35,0.05);border-radius:3px}.goost .markdown-body code br,.goost .markdown-body tt br{display:none}.goost .markdown-body del code{text-decoration:inherit}.goost .markdown-body pre{word-wrap:normal}.goost .markdown-body pre>code{padding:0;margin:0;font-size:100%;word-break:normal;white-space:pre;background:transparent;border:0}.goost .markdown-body .highlight{margin-bottom:16px}.goost .markdown-body .highlight pre{margin-bottom:0;word-break:normal}.goost .markdown-body .highlight pre,.goost .markdown-body pre{padding:16px;overflow:auto;font-size:85%;line-height:1.45;background-color:#f6f8fa;border-radius:3px}.goost .markdown-body pre code,.goost .markdown-body pre tt{display:inline;max-width:auto;padding:0;margin:0;overflow:visible;line-height:inherit;word-wrap:normal;background-color:transparent;border:0}.goost .markdown-body .csv-data td,.goost .markdown-body .csv-data th{padding:5px;overflow:hidden;font-size:12px;line-height:1;text-align:left;white-space:nowrap}.goost .markdown-body .csv-data .blob-num{padding:10px 8px 9px;text-align:right;background:#fff;border:0}.goost .markdown-body .csv-data tr{border-top:0}.goost .markdown-body .csv-data th{font-weight:600;background:#f6f8fa;border-top:0}.goost .pl-c{color:#6a737d}.goost .pl-c1,.goost .pl-s .pl-v{color:#005cc5}.goost .pl-e,.goost .pl-en{color:#6f42c1}.goost .pl-smi,.goost .pl-s .pl-s1{color:#24292e}.goost .pl-ent{color:#22863a}.goost .pl-k{color:#d73a49}.goost .pl-s,.goost .pl-pds,.goost .pl-s .pl-pse .pl-s1,.goost .pl-sr,.goost .pl-sr .pl-cce,.goost .pl-sr .pl-sre,.goost .pl-sr .pl-sra{color:#032f62}.goost .pl-v,.goost .pl-smw{color:#e36209}.goost .pl-bu{color:#b31d28}.goost .pl-ii{color:#fafbfc;background-color:#b31d28}.goost .pl-c2{color:#fafbfc;background-color:#d73a49}.goost .pl-c2::before{content:"^M"}.goost .pl-sr .pl-cce{font-weight:bold;color:#22863a}.goost .pl-ml{color:#735c0f}.goost .pl-mh,.goost .pl-mh .pl-en,.goost .pl-ms{font-weight:bold;color:#005cc5}.goost .pl-mi{font-style:italic;color:#24292e}.goost .pl-mb{font-weight:bold;color:#24292e}.goost .pl-md{color:#b31d28;background-color:#ffeef0}.goost .pl-mi1{color:#22863a;background-color:#f0fff4}.goost .pl-mc{color:#e36209;background-color:#ffebda}.goost .pl-mi2{color:#f6f8fa;background-color:#005cc5}.goost .pl-mdr{font-weight:bold;color:#6f42c1}.goost .pl-ba{color:#586069}.goost .pl-sg{color:#959da5}.goost .pl-corl{text-decoration:underline;color:#032f62}.goost .breadcrumb{margin-bottom:10px;font-size:18px;color:#586069}.goost .breadcrumb .separator::before,.goost .breadcrumb .separator::after{content:" "}.goost .breadcrumb strong.final-path{color:#24292e}.goost .breadcrumb .zeroclipboard-button{display:inline-block;margin-left:5px}.goost .breadcrumb .repo-root{font-weight:600}.goost .breadcrumb .octicon{vertical-align:-2px}.goost .editor-license-template,.goost .editor-code-of-conduct-template,.goost .editor-gitignore-template{position:relative;top:3px;display:block;float:right;font-size:14px}.goost .editor-license-template .select-menu-git-ignore,.goost .editor-code-of-conduct-template .select-menu-git-ignore,.goost .editor-gitignore-template .select-menu-git-ignore{right:0}.goost .editor-abort{display:inline;font-size:14px}.goost .blob-interaction-bar{position:relative;background-color:#f2f2f2;border-bottom:1px solid #e5e5e5}.goost .blob-interaction-bar::before{display:table;content:""}.goost .blob-interaction-bar::after{display:table;clear:both;content:""}.goost .blob-interaction-bar .octicon-search{position:absolute;top:10px;left:10px;font-size:12px;color:#586069}.goost .blob-filter{width:100%;padding:4px 20px 5px 30px;font-size:12px;border:0;border-radius:0;outline:none}.goost .blob-filter:focus{outline:none}.goost .html-blob{margin-bottom:15px}.goost .license-summary-octicon{color:#959da5}.goost .rule-type-permissions{color:#28a745}.goost .rule-type-conditions{color:#0366d6}.goost .rule-type-limitations{color:#d73a49}.goost .blob-wrapper{overflow-x:auto;overflow-y:hidden;border-bottom-right-radius:3px;border-bottom-left-radius:3px}.goost .blob-wrapper-embedded{max-height:240px;overflow-y:auto}.goost .diff-table{width:100%;border-collapse:separate}.goost .diff-table .line-comments{padding:10px;vertical-align:top;border-top:1px solid #e1e4e8}.goost .diff-table .line-comments:first-child+.empty-cell{border-left-width:1px}.goost .diff-table tr:not(:last-child) .line-comments{border-top:1px solid #e1e4e8;border-bottom:1px solid #e1e4e8}.goost .blob-num{width:1%;min-width:50px;padding-right:10px;padding-left:10px;font-family:"SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace;font-size:12px;line-height:20px;color:rgba(27,31,35,0.3);text-align:right;white-space:nowrap;vertical-align:top;cursor:pointer;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}.goost .blob-num:hover{color:rgba(27,31,35,0.6)}.goost .blob-num::before{content:attr(data-line-number)}.goost .blob-num.non-expandable{cursor:default}.goost .blob-num.non-expandable:hover{color:rgba(27,31,35,0.3)}.goost .blob-code{position:relative;padding-right:10px;padding-left:10px;line-height:20px;vertical-align:top}.goost .blob-code-inner{overflow:visible;font-family:"SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace;font-size:12px;color:#24292e;word-wrap:normal;white-space:pre}.goost .blob-code-inner .x-first{border-top-left-radius:0.2em;border-bottom-left-radius:0.2em}.goost .blob-code-inner .x-last{border-top-right-radius:0.2em;border-bottom-right-radius:0.2em}.goost .blob-code-inner::before{content:""}.goost .blob-code-inner.highlighted{background-color:#fffbdd}.goost .soft-wrap .diff-table{table-layout:fixed}.goost .soft-wrap .blob-code{padding-left:18px;text-indent:-7px}.goost .soft-wrap .blob-code-inner{word-wrap:break-word;white-space:pre-wrap}.goost .soft-wrap .no-nl-marker{display:none}.goost .soft-wrap .add-line-comment{margin-left:-28px}.goost .blob-num-hunk,.goost .blob-code-hunk,.goost .blob-num-expandable,.goost .blob-code-expandable{color:rgba(27,31,35,0.5);vertical-align:middle}.goost .blob-num-hunk,.goost .blob-num-expandable{background-color:#dbedff}.goost .blob-code-hunk,.goost .blob-code-expandable{padding-top:4px;padding-bottom:4px;background-color:#f1f8ff;border-width:1px 0}.goost .blob-expanded .blob-num,.goost .blob-expanded .blob-code{background-color:#fafbfc}.goost .blob-expanded+tr:not(.blob-expanded) .blob-num,.goost .blob-expanded+tr:not(.blob-expanded) .blob-code{border-top:1px solid #eaecef}.goost .blob-expanded .blob-num-hunk{border-top:1px solid #eaecef}.goost tr:not(.blob-expanded)+.blob-expanded .blob-num,.goost tr:not(.blob-expanded)+.blob-expanded .blob-code{border-top:1px solid #eaecef}.goost .blob-num-expandable{padding:0;font-size:12px;text-align:center}.goost .blob-num-expandable .octicon{vertical-align:top}.goost .blob-num-expandable .diff-expander{display:block;width:auto;height:auto;padding:4px 11px 4px 10px;margin-right:-1px;color:#586069;cursor:pointer}.goost .blob-num-expandable .diff-expander:hover{color:#fff;text-shadow:none;background-color:#0366d6;border-color:#0366d6}.goost .blob-code-addition{background-color:#e6ffed}.goost .blob-code-addition .x{color:#24292e;background-color:#acf2bd}.goost .blob-num-addition{background-color:#cdffd8;border-color:#bef5cb}.goost .blob-code-deletion{background-color:#ffeef0}.goost .blob-code-deletion .x{color:#24292e;background-color:#fdb8c0}.goost .blob-num-deletion{background-color:#ffdce0;border-color:#fdaeb7}.goost .selected-line.blob-code{background-color:#fffbdd}.goost .selected-line.blob-code .x{background-color:transparent}.goost .selected-line.blob-num{background-color:#fff5b1;border-color:#ffea7f}.goost .add-line-comment{position:relative;z-index:5;float:left;width:22px;height:22px;margin:-2px -10px -2px -20px;line-height:21px;color:#fff;text-align:center;text-indent:0;cursor:pointer;background-color:#0366d6;background-image:linear-gradient(#0372ef, #0366d6);border-radius:3px;box-shadow:0 1px 4px rgba(27,31,35,0.15);opacity:0;transition:-webkit-transform 0.1s ease-in-out;transition:transform 0.1s ease-in-out;-webkit-transform:scale(0.8, 0.8);transform:scale(0.8, 0.8)}.goost .add-line-comment:hover{-webkit-transform:scale(1, 1);transform:scale(1, 1)}.is-hovered .goost .add-line-comment,.goost .add-line-comment:focus{opacity:1}.goost .add-line-comment .octicon{vertical-align:text-top;pointer-events:none}.goost .add-line-comment.octicon-check{background:#333;opacity:1}.goost .inline-comment-form{border:1px solid #dfe2e5;border-radius:3px}.goost .inline-review-comment{margin-top:0 !important;margin-bottom:10px !important}.goost .inline-review-comment .gc:first-child+tr .blob-num,.goost .inline-review-comment .gc:first-child+tr .blob-code{padding-top:5px}.goost .inline-review-comment tr:last-child{border-bottom-right-radius:2px;border-bottom-left-radius:2px}.goost .inline-review-comment tr:last-child .blob-num,.goost .inline-review-comment tr:last-child .blob-code{padding-bottom:8px}.goost .inline-review-comment tr:last-child .blob-num:first-child,.goost .inline-review-comment tr:last-child .blob-code:first-child{border-bottom-left-radius:2px}.goost .inline-review-comment tr:last-child .blob-num:last-child,.goost .inline-review-comment tr:last-child .blob-code:last-child{border-bottom-right-radius:2px}.goost .timeline-inline-comments{width:100%;table-layout:fixed}.goost .timeline-inline-comments .inline-comments,.goost .show-inline-notes .inline-comments{display:table-row}.goost .inline-comments{display:none}.goost .inline-comments.is-collapsed{display:none}.goost .inline-comments .line-comments.is-collapsed{visibility:hidden}.goost .inline-comments .line-comments+.blob-num{border-left-width:1px}.goost .inline-comments .timeline-comment{margin-bottom:10px}.goost .inline-comments .inline-comment-form,.goost .inline-comments .inline-comment-form-container{max-width:780px}.goost .comment-holder{max-width:780px}.goost .line-comments+.line-comments,.goost .empty-cell+.line-comments{border-left:1px solid #eaecef}.goost .inline-comment-form-container .inline-comment-form,.goost .inline-comment-form-container.open .inline-comment-form-actions{display:none}.goost .inline-comment-form-container .inline-comment-form-actions,.goost .inline-comment-form-container.open .inline-comment-form{display:block}.goost body.split-diff .container,.goost body.full-width .container{width:100%;padding-right:20px;padding-left:20px}.goost body.split-diff .repository-content,.goost body.full-width .repository-content{width:100%}.goost body.split-diff .new-pr-form,.goost body.full-width .new-pr-form{max-width:980px}.goost body.split-diff .new-pr-form .discussion-sidebar,.goost body.full-width .new-pr-form .discussion-sidebar{width:200px}.goost .file-diff-split{table-layout:fixed}.goost .file-diff-split .blob-code+.blob-num{border-left:1px solid #f6f8fa}.goost .file-diff-split .blob-code-inner{word-wrap:break-word;white-space:pre-wrap}.goost .file-diff-split .empty-cell{cursor:default;background-color:#fafbfc;border-right-color:#eaecef}.goost .submodule-diff-stats .octicon-diff-removed{color:#cb2431}.goost .submodule-diff-stats .octicon-diff-renamed{color:#677a85}.goost .submodule-diff-stats .octicon-diff-modified{color:#d0b44c}.goost .submodule-diff-stats .octicon-diff-added{color:#28a745}.goost .BlobToolbar{left:-17px}.goost .BlobToolbar-dropdown{margin-left:-2px}.goost .task-list-item{list-style-type:none}.goost .task-list-item label{font-weight:normal}.goost .task-list-item.enabled label{cursor:pointer}.goost .task-list-item+.task-list-item{margin-top:3px}.goost .task-list-item .handle{display:none}.goost .task-list-item-checkbox{margin:0 0.2em 0.25em -1.6em;vertical-align:middle}.goost .reorderable-task-lists .markdown-body .contains-task-list{padding:0}.goost .reorderable-task-lists .markdown-body li:not(.task-list-item){margin-left:26px}.goost .reorderable-task-lists .markdown-body ol:not(.contains-task-list) li,.goost .reorderable-task-lists .markdown-body ul:not(.contains-task-list) li{margin-left:0}.goost .reorderable-task-lists .markdown-body li p{margin-top:0}.goost .reorderable-task-lists .markdown-body .task-list-item{padding-right:15px;padding-left:42px;margin-right:-15px;margin-left:-15px;border:1px solid transparent}.goost .reorderable-task-lists .markdown-body .task-list-item+.task-list-item{margin-top:0}.goost .reorderable-task-lists .markdown-body .task-list-item .contains-task-list{padding-top:4px}.goost .reorderable-task-lists .markdown-body .task-list-item .handle{display:block;float:left;width:20px;padding:2px 0 0 2px;margin-left:-43px;opacity:0}.goost .reorderable-task-lists .markdown-body .task-list-item .drag-handle{fill:#333}.goost .reorderable-task-lists .markdown-body .task-list-item.hovered{background:#fafafa;border-top-color:#ededed;border-bottom-color:#ededed}.goost .reorderable-task-lists .markdown-body .task-list-item.hovered>.handle{opacity:1}.goost .reorderable-task-lists .markdown-body .task-list-item.is-dragging{opacity:0}.goost .reorderable-task-lists .markdown-body .task-list-item.is-ghost{border-right-color:#ededed;border-left-color:#ededed}.goost .review-comment-contents .markdown-body .task-list-item{padding-left:42px;margin-right:-12px;margin-left:-12px;border-top-left-radius:3px;border-bottom-left-radius:3px}.goost .review-comment-contents .markdown-body .task-list-item.hovered{border-left-color:#ededed}.goost .highlight{padding:0;margin:0;font-family:"SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace;font-size:12px;font-weight:normal;line-height:1.4;color:#333;background:#fff;border:0}.goost .render-viewer-error,.goost .render-viewer-fatal,.goost .render-viewer-invalid,.goost .octospinner{display:none}.goost iframe.render-viewer{width:100%;height:480px;overflow:hidden;border:0}.goost pre,.goost code{font-family:"SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace !important;white-space:pre}.goost .goost-meta{padding:10px;overflow:hidden;font:12px -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol";color:#586069;background-color:#f7f7f7;border-radius:0 0 2px 2px}.goost .goost-meta a{font-weight:600;color:#666;text-decoration:none;border:0}.goost .goost-data{overflow:auto;word-wrap:normal;background-color:#fff;border-bottom:1px solid #ddd;border-radius:2px 2px 0 0}.goost .goost-file{margin-bottom:1em;font-family:"SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace;border:1px solid #ddd;border-bottom:1px solid #ccc;border-radius:3px}.goost .goost-file article{padding:6px}.goost .goost-file .scroll .goost-data{position:absolute;top:0;right:0;bottom:30px;left:0;overflow:scroll}.goost .goost-file .scroll .goost-meta{position:absolute;right:0;bottom:0;left:0}.goost .blob-num{min-width:inherit;padding:1px 10px !important;background:transparent}.goost .blob-code{padding:1px 10px !important;text-align:left;background:transparent;border:0}.goost .blob-wrapper table{border-collapse:collapse}.goost .blob-wrapper tr:first-child td{padding-top:4px}.goost .markdown-body .anchor{display:none}
.goost .goost-file {border:none}
.goost .goost-data {border-bottom:none}
.goost .markdown-body .highlight pre, .goost .markdown-body pre {font-size:0.85rem}
</style>
<div id="goost83245689" class="goost">
<div class="goost-file">
<div class="goost-data">
<div class="js-goost-file-update-container js-task-list-container file-box">
<div id="file-ch01-md" class="file">
<div id="readme" class="readme blob instapaper_body">
<article class="markdown-body entry-content" itemprop="text"><p>This is chapter 1 in a <a href="https://github.com/japgolly/learning/tree/public/recursion-blog">series of blog posts about recursion schemes</a>.
This series uses Scala, and will focus on usage and applicability.
It will be scarce in theory, and abundant in examples.
The theory is valuable and fascinating, but I often find that knowing the theory alone is only half understanding.
The other half of understanding comes from, and enables, practical application.
I recommend you bounce back and forth between this series and theory.
The internet is rich with blogs and videos on theory that explain it better than I would, so use those awesome resources.</p>
<h2><a href="#overview" aria-hidden="true" class="anchor" id="user-content-overview"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Overview</h2>
<p>The goal of this post is to prepare your data types so that you can abstrct
over/away recursion, and be able to use all the generalisations that we'll
explore in all future chapters in this series.</p>
<p>In order to prepare your data, there are three things to do:</p>
<ol>
<li>Make the recursive positions in your data type abstract</li>
<li>Create a <code>Functor</code> for your data type</li>
<li>Wraps your data type in <code>Fix[_]</code></li>
</ol>
<h2><a href="#step-1-remove-recursion-from-the-data-type" aria-hidden="true" class="anchor" id="user-content-step-1-remove-recursion-from-the-data-type"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Step 1. Remove recursion from the data type</h2>
<p>Add a type parameter to your data type that will be used to represent recursion
(and other things as you'll see in future chapters). Everywhere that your type references
itself, replace the type with the new abstract type parameter.</p>
<h3><a href="#intlist" aria-hidden="true" class="anchor" id="user-content-intlist"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><code>IntList</code></h3>
<p>Example #1: let's say you have your own hand-written cons-list, specialised to <code>Int</code>;
maybe you want to avoid boxing with <code>List[Int]</code>. It might look like this:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-c"><span class="pl-c">//</span> Before</span>
<span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">IntList</span>
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">IntCons</span>(<span class="pl-v">head</span>: <span class="pl-k">Int</span>, <span class="pl-v">tail</span>: <span class="pl-en">IntList</span>) <span class="pl-k">extends</span> <span class="pl-e">IntList</span>
<span class="pl-k">case</span> <span class="pl-k">object</span> <span class="pl-en">IntNil</span> <span class="pl-k">extends</span> <span class="pl-e">IntList</span></pre></div>
<p>This type references itself in <code>IntCons#tail</code>. Let's fix that...</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-c"><span class="pl-c">//</span> After (v1)</span>
<span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">IntList</span>[<span class="pl-en">F</span>]
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">IntCons</span>[<span class="pl-en">F</span>](<span class="pl-v">head</span>: <span class="pl-k">Int</span>, <span class="pl-v">tail</span>: <span class="pl-en">F</span>) <span class="pl-k">extends</span> <span class="pl-e">IntList</span>[<span class="pl-en">F</span>]
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">IntNil</span>[<span class="pl-en">F</span>]() <span class="pl-k">extends</span> <span class="pl-e">IntList</span>[<span class="pl-en">F</span>]</pre></div>
<p>Ok so now it never references itself, but we've had to modify the non-recursive case,
<code>IntNil</code>, from an object to a <code>case class</code> with zero params.
I'd prefer <code>IntNil</code> to remain an object for better ergonomics and efficiency so
let's use variance.</p>
<p><em>(Note: Many Scala FP'ers avoid type variance entirely because it's
kind of broken in theory, can lead to bugs in higher-kinded contexts, screws up
implicits sometimes and breaks type inference sometimes too. I advise:
learn about it, understand the tradeoffs and decide on a case-by-case basis.)</em></p>
<div class="highlight highlight-source-scala"><pre><span class="pl-c"><span class="pl-c">//</span> After (v2)</span>
<span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">IntList</span>[<span class="pl-k">+</span><span class="pl-en">F</span>]
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">IntCons</span>[<span class="pl-k">+</span><span class="pl-en">F</span>](<span class="pl-v">head</span>: <span class="pl-k">Int</span>, <span class="pl-v">tail</span>: <span class="pl-en">F</span>) <span class="pl-k">extends</span> <span class="pl-e">IntList</span>[<span class="pl-en">F</span>]
<span class="pl-k">case</span> <span class="pl-k">object</span> <span class="pl-en">IntNil</span> <span class="pl-k">extends</span> <span class="pl-e">IntList</span>[<span class="pl-en">Nothing</span>]</pre></div>
<h3><a href="#binarytree" aria-hidden="true" class="anchor" id="user-content-binarytree"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><code>BinaryTree</code></h3>
<p>How about a binary tree with an abstract type <code>A</code> in the nodes:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-c"><span class="pl-c">//</span> Before</span>
<span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">BinaryTree</span>[<span class="pl-k">+</span><span class="pl-en">A</span>]
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Node</span>[<span class="pl-k">+</span><span class="pl-en">A</span>](<span class="pl-v">left</span>: <span class="pl-en">BinaryTree</span>[<span class="pl-en">A</span>], <span class="pl-v">value</span>: <span class="pl-en">A</span>, <span class="pl-v">right</span>: <span class="pl-en">BinaryTree</span>[<span class="pl-en">A</span>]) <span class="pl-k">extends</span> <span class="pl-e">BinaryTree</span>[<span class="pl-en">A</span>]
<span class="pl-k">case</span> <span class="pl-k">object</span> <span class="pl-en">Leaf</span> <span class="pl-k">extends</span> <span class="pl-e">BinaryTree</span>[<span class="pl-en">Nothing</span>]</pre></div>
<p>Here we replace the left and right branches.</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-c"><span class="pl-c">//</span> After</span>
<span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">BinaryTree</span>[<span class="pl-k">+</span><span class="pl-en">A</span>, <span class="pl-k">+</span><span class="pl-en">F</span>]
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Node</span>[<span class="pl-k">+</span><span class="pl-en">A</span>, <span class="pl-k">+</span><span class="pl-en">F</span>](<span class="pl-v">left</span>: <span class="pl-en">F</span>, <span class="pl-v">value</span>: <span class="pl-en">A</span>, <span class="pl-v">right</span>: <span class="pl-en">F</span>) <span class="pl-k">extends</span> <span class="pl-e">BinaryTree</span>[<span class="pl-en">A</span>, <span class="pl-en">F</span>]
<span class="pl-k">case</span> <span class="pl-k">object</span> <span class="pl-en">Leaf</span> <span class="pl-k">extends</span> <span class="pl-e">BinaryTree</span>[<span class="pl-en">Nothing</span>, <span class="pl-en">Nothing</span>]</pre></div>
<p>Don't worry about preserving the <code>A</code> in the branches -- don't try to make it an <code>F[A]</code> -- just a plain
old <code>*</code>-kinded <code>F</code> is all you need. Step 3 will ensure that <code>A</code>s persist
in both branches' children.</p>
<h3><a href="#json" aria-hidden="true" class="anchor" id="user-content-json"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>JSON</h3>
<p>JSON is recursive too.</p>
<p>JSON has arrays of JSON, it has objects with JSON values,
those values can be arrays that contains even more objects with nested arrays and...
you get the picture.</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-c"><span class="pl-c">//</span> Before</span>
<span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">Json</span>
<span class="pl-k">object</span> <span class="pl-en">Json</span> {
<span class="pl-k">case</span> <span class="pl-k">object</span> <span class="pl-en">Null</span> <span class="pl-k">extends</span> <span class="pl-e">Json</span>
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Bool</span>(<span class="pl-v">value</span>: <span class="pl-k">Boolean</span>) <span class="pl-k">extends</span> <span class="pl-e">Json</span>
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Str</span> (<span class="pl-v">value</span>: <span class="pl-k">String</span>) <span class="pl-k">extends</span> <span class="pl-e">Json</span>
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Num</span> (<span class="pl-v">value</span>: <span class="pl-k">Double</span>) <span class="pl-k">extends</span> <span class="pl-e">Json</span>
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Arr</span> (<span class="pl-v">values</span>: <span class="pl-en">List</span>[<span class="pl-en">Json</span>]) <span class="pl-k">extends</span> <span class="pl-e">Json</span>
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Obj</span> (<span class="pl-v">fields</span>: <span class="pl-en">List</span>[(<span class="pl-k">String</span>, <span class="pl-en">Json</span>)]) <span class="pl-k">extends</span> <span class="pl-e">Json</span>
}</pre></div>
<p>Only replace the self references; preserve the outer type.
<code>List[Json]</code> should be <code>List[F]</code>, not <code>F</code>.</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-c"><span class="pl-c">//</span> After</span>
<span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">Json</span>[<span class="pl-k">+</span><span class="pl-en">F</span>]
<span class="pl-k">object</span> <span class="pl-en">Json</span> {
<span class="pl-k">case</span> <span class="pl-k">object</span> <span class="pl-en">Null</span> <span class="pl-k">extends</span> <span class="pl-e">Json</span>
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Bool</span> (<span class="pl-v">value</span>: <span class="pl-k">Boolean</span>) <span class="pl-k">extends</span> <span class="pl-e">Json</span>[<span class="pl-en">Nothing</span>]
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Str</span> (<span class="pl-v">value</span>: <span class="pl-k">String</span>) <span class="pl-k">extends</span> <span class="pl-e">Json</span>[<span class="pl-en">Nothing</span>]
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Num</span> (<span class="pl-v">value</span>: <span class="pl-k">Double</span>) <span class="pl-k">extends</span> <span class="pl-e">Json</span>[<span class="pl-en">Nothing</span>]
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Arr</span>[<span class="pl-en">F</span>](<span class="pl-v">values</span>: <span class="pl-en">List</span>[<span class="pl-en">F</span>]) <span class="pl-k">extends</span> <span class="pl-e">Json</span>[<span class="pl-en">F</span>]
<span class="pl-k">final</span> <span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Obj</span>[<span class="pl-en">F</span>](<span class="pl-v">fields</span>: <span class="pl-en">List</span>[(<span class="pl-k">String</span>, <span class="pl-en">F</span>)]) <span class="pl-k">extends</span> <span class="pl-e">Json</span>[<span class="pl-en">F</span>]
}</pre></div>
<h1><a href="#step-2-create-a-functor" aria-hidden="true" class="anchor" id="user-content-step-2-create-a-functor"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Step 2. Create a Functor</h1>
<p>In this step we create a <code>Functor</code> for our data types.
<code>Functor</code> is a type class that exists in the FP lib of your choice, namely
Scalaz or Cats. It looks like this:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">trait</span> <span class="pl-en">Functor</span>[<span class="pl-en">F</span>[_]] {
<span class="pl-k">def</span> <span class="pl-en">map</span>[<span class="pl-en">A</span>, <span class="pl-en">B</span>](<span class="pl-v">fa</span>: <span class="pl-en">F</span>[<span class="pl-en">A</span>])(<span class="pl-v">f</span>: <span class="pl-en">A</span> <span class="pl-k">=></span> <span class="pl-en">B</span>)<span class="pl-k">:</span> <span class="pl-en">F</span>[<span class="pl-en">B</span>]
}</pre></div>
<p>It empowers the world outside your data structure to change one of its abstract
types, and by necessity, all values of that type. It's just like calling <code>.map</code> on a list:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-c"><span class="pl-c">//</span> Change the values</span>
<span class="pl-en">List</span>(<span class="pl-c1">1</span>, <span class="pl-c1">2</span>, <span class="pl-c1">3</span>).map(_ <span class="pl-k">*</span> <span class="pl-c1">10</span>)
<span class="pl-c"><span class="pl-c">//</span> yields</span>
<span class="pl-en">List</span>[<span class="pl-k">Int</span>](<span class="pl-c1">10</span>, <span class="pl-c1">20</span>, <span class="pl-c1">30</span>)
<span class="pl-c"><span class="pl-c">//</span> Change the type too</span>
<span class="pl-en">List</span>(<span class="pl-c1">1</span>, <span class="pl-c1">2</span>, <span class="pl-c1">3</span>).map(i <span class="pl-k">=></span> s<span class="pl-s"><span class="pl-pds">"</span>[$i]<span class="pl-pds">"</span></span>)
<span class="pl-c"><span class="pl-c">//</span> yields</span>
<span class="pl-en">List</span>[<span class="pl-k">String</span>](<span class="pl-s"><span class="pl-pds">"</span>[1]<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>[2]<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>[3]<span class="pl-pds">"</span></span>)</pre></div>
<p>This functor is how the generic recursion abstractions can have access to, and
control over the recursive spots in your data.</p>
<p><em>(Note: If you want to use monadic variants of morphisms later, you'll need to
upgrade your Functor to a Traverse. I'll show that in a future post but just
keep it in mind.)</em></p>
<p>Let's create instances for our examples above.
It doesn't matter if you use Scalaz or Cats, only the imports need to change.
The code itself is identical.</p>
<h3><a href="#intlist-1" aria-hidden="true" class="anchor" id="user-content-intlist-1"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><code>IntList</code></h3>
<p>Let the compiler guide you, it will only accept one implementation:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">implicit</span> <span class="pl-k">val</span> <span class="pl-en">functor</span><span class="pl-k">:</span> <span class="pl-en">Functor</span>[<span class="pl-en">IntList</span>] <span class="pl-k">=</span> <span class="pl-k">new</span> <span class="pl-en">Functor</span>[<span class="pl-en">IntList</span>] {
<span class="pl-k">override</span> <span class="pl-k">def</span> <span class="pl-en">map</span>[<span class="pl-en">A</span>, <span class="pl-en">B</span>](<span class="pl-v">fa</span>: <span class="pl-en">IntList</span>[<span class="pl-en">A</span>])(<span class="pl-v">f</span>: <span class="pl-en">A</span> <span class="pl-k">=></span> <span class="pl-en">B</span>)<span class="pl-k">:</span> <span class="pl-en">IntList</span>[<span class="pl-en">B</span>] <span class="pl-k">=</span> fa <span class="pl-k">match</span> {
<span class="pl-k">case</span> <span class="pl-en">IntCons</span>(head, tail) <span class="pl-k">=></span> <span class="pl-en">IntCons</span>(head, f(tail))
<span class="pl-k">case</span> <span class="pl-en">IntNil</span> <span class="pl-k">=></span> <span class="pl-en">IntNil</span>
}
}</pre></div>
<p>Note: You could also use an <code>implicit object</code> but it can cause implicit
resolution problems later. I can't remember why/when anymore, it's just become habit.</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-c"><span class="pl-c">//</span> Also possible but leads to implicit resolution problems</span>
<span class="pl-k">implicit</span> <span class="pl-k">object</span> <span class="pl-en">IntListFunctor</span> <span class="pl-k">extends</span> <span class="pl-e">Functor</span>[<span class="pl-en">IntList</span>] {
<span class="pl-k">override</span> <span class="pl-k">def</span> <span class="pl-en">map</span>[<span class="pl-en">A</span>, <span class="pl-en">B</span>](<span class="pl-v">fa</span>: <span class="pl-en">IntList</span>[<span class="pl-en">A</span>])(<span class="pl-v">f</span>: <span class="pl-en">A</span> <span class="pl-k">=></span> <span class="pl-en">B</span>)<span class="pl-k">:</span> <span class="pl-en">IntList</span>[<span class="pl-en">B</span>] <span class="pl-k">=</span> fa <span class="pl-k">match</span> {
<span class="pl-k">case</span> <span class="pl-en">IntCons</span>(head, tail) <span class="pl-k">=></span> <span class="pl-en">IntCons</span>(head, f(tail))
<span class="pl-k">case</span> <span class="pl-en">IntNil</span> <span class="pl-k">=></span> <span class="pl-en">IntNil</span>
}
}</pre></div>
<p>Another side-note, do explicitly annotate the type.</p>
<ol>
<li>If you don't the type will be a structural type which will slow down the compiler and mess with implicit resolution.</li>
<li>Explicit annotation will be mandatory in Scala v3 anyway.</li>
</ol>
<div class="highlight highlight-source-scala"><pre><span class="pl-c"><span class="pl-c">//</span> Don't do this</span>
<span class="pl-k">implicit</span> <span class="pl-k">val</span> <span class="pl-en">functor</span> <span class="pl-k">=</span> <span class="pl-k">new</span> <span class="pl-en">Functor</span>[<span class="pl-en">IntList</span>] {</pre></div>
<h3><a href="#binarytree-1" aria-hidden="true" class="anchor" id="user-content-binarytree-1"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><code>BinaryTree</code></h3>
<p>As intended, this next case introduces a bit more complexity.
We want to provide the ability to transform the recursive positions which means
we need to keep the <code>value: A</code> position abstract and stable.</p>
<p>You'll need to use <a href="https://github.com/non/kind-projector">kind-projector</a>
to get the nice <code>BinaryTree[A, ?]</code> syntax, instead of the monstrous,
out-of-the-box <code>({ type L[X] = BinaryTree[A, X] })#L</code> syntax.
If types and terms weren't so different, it'd be <code>BinaryTree[A, _]</code> just like
the underscore in <code>List(1,2,3).map(_ * 100)</code>.</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">implicit</span> <span class="pl-k">def</span> <span class="pl-en">functor</span>[<span class="pl-en">A</span>]<span class="pl-k">:</span> <span class="pl-en">Functor</span>[<span class="pl-en">BinaryTree</span>[<span class="pl-en">A</span>, <span class="pl-k">?</span>]] <span class="pl-k">=</span> <span class="pl-k">new</span> <span class="pl-en">Functor</span>[<span class="pl-en">BinaryTree</span>[<span class="pl-en">A</span>, <span class="pl-k">?</span>]] {
<span class="pl-k">override</span> <span class="pl-k">def</span> <span class="pl-en">map</span>[<span class="pl-en">B</span>, <span class="pl-en">C</span>](<span class="pl-v">fa</span>: <span class="pl-en">BinaryTree</span>[<span class="pl-en">A</span>, <span class="pl-en">B</span>])(<span class="pl-v">f</span>: <span class="pl-en">B</span> <span class="pl-k">=></span> <span class="pl-en">C</span>)<span class="pl-k">:</span> <span class="pl-en">BinaryTree</span>[<span class="pl-en">A</span>, <span class="pl-en">C</span>] <span class="pl-k">=</span> fa <span class="pl-k">match</span> {
<span class="pl-k">case</span> <span class="pl-en">Node</span>(left, value, right) <span class="pl-k">=></span> <span class="pl-en">Node</span>(f(left), value, f(right))
<span class="pl-k">case</span> <span class="pl-en">Leaf</span> <span class="pl-k">=></span> <span class="pl-en">Leaf</span>
}
}</pre></div>
<p>There's nothing special about the <code>F</code> type over the <code>A</code>. You could also write a
functor over the <code>A</code> type, i.e. a <code>Functor[BinaryTree[?, F]]</code> instance.</p>
<h3><a href="#json-1" aria-hidden="true" class="anchor" id="user-content-json-1"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>JSON</h3>
<p>Here our <code>F</code>s are embedded in other values so we have to work a little harder to
transform them, but not much harder.
Lists have functors too so it's easy peasy: just call <code>.map</code>.</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">implicit</span> <span class="pl-k">val</span> <span class="pl-en">functor</span><span class="pl-k">:</span> <span class="pl-en">Functor</span>[<span class="pl-en">Json</span>] <span class="pl-k">=</span> <span class="pl-k">new</span> <span class="pl-en">Functor</span>[<span class="pl-en">Json</span>] {
<span class="pl-k">override</span> <span class="pl-k">def</span> <span class="pl-en">map</span>[<span class="pl-en">A</span>, <span class="pl-en">B</span>](<span class="pl-v">fa</span>: <span class="pl-en">Json</span>[<span class="pl-en">A</span>])(<span class="pl-v">f</span>: <span class="pl-en">A</span> <span class="pl-k">=></span> <span class="pl-en">B</span>)<span class="pl-k">:</span> <span class="pl-en">Json</span>[<span class="pl-en">B</span>] <span class="pl-k">=</span> fa <span class="pl-k">match</span> {
<span class="pl-k">case</span> <span class="pl-en">Null</span> <span class="pl-k">=></span> <span class="pl-en">Null</span>
<span class="pl-k">case</span> <span class="pl-v">j</span>: <span class="pl-en">Bool</span> <span class="pl-k">=></span> j
<span class="pl-k">case</span> <span class="pl-v">j</span>: <span class="pl-en">Str</span> <span class="pl-k">=></span> j
<span class="pl-k">case</span> <span class="pl-v">j</span>: <span class="pl-en">Num</span> <span class="pl-k">=></span> j
<span class="pl-k">case</span> <span class="pl-en">Arr</span>(values) <span class="pl-k">=></span> <span class="pl-en">Arr</span>(values.map(f))
<span class="pl-k">case</span> <span class="pl-en">Obj</span>(fields) <span class="pl-k">=></span> <span class="pl-en">Obj</span>(fields.map { <span class="pl-k">case</span> (k, v) <span class="pl-k">=></span> (k, f(v)) })
}
}</pre></div>
<h1><a href="#step-3-the-fixpoint" aria-hidden="true" class="anchor" id="user-content-step-3-the-fixpoint"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Step 3. The Fixpoint</h1>
<p>In step 1, we went from a recursive data structure to a non-recursive data structure.
Our goal is to be able to abstract over/away recursion, not to vanquish it.
How do we regain our recursion?
After all, we still want the users of our amazing <code>IntList</code> library to be able to
store more than one element!</p>
<p>We've going to wrap our types in a magical fixpoint type.
Doing so will give us back our recursion.</p>
<p>Here's a definition of Fix:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">case</span> <span class="pl-k">class</span> <span class="pl-en">Fix</span>[<span class="pl-en">F</span>[_]](<span class="pl-v">unfix</span>: <span class="pl-en">F</span>[<span class="pl-en">Fix</span>[<span class="pl-en">F</span>]])</pre></div>
<p>Confused? This isn't a theory series but get a pen and paper, plug in <code>IntList</code> and expand the alias step-by-step; it'll clear it up real quick.</p>
<p><em>Exciting side note: <a href="https://twitter.com/S11001001" rel="nofollow">Stephen Compall</a> and <a href="https://twitter.com/tomas_mikula" rel="nofollow">Tomas Mikula</a> found a way to define <code>Fix</code> without boxing!
See <a href="https://github.com/scalaz/scalaz/pull/1472/files">how, here</a>.
I measured a <a href="https://github.com/japgolly/microlibs-scala/commit/c2c1f8adaa9c9166ed6c10a0c3c09cfdbc0c8d58">20-30% improvement</a> and copied the approach in <a href="https://github.com/japgolly/microlibs-scala/tree/master/recursion">my own recursion library</a>.
Very awesome stuff.</em></p>
<p>Anyway, wrap your data types in <code>Fix[_]</code> and you get your recursion back.</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">type</span> <span class="pl-en">RecursiveIntList</span> <span class="pl-k">=</span> <span class="pl-en">Fix</span>[<span class="pl-en">IntList</span>]
<span class="pl-k">type</span> <span class="pl-en">RecursiveBinaryTree</span>[<span class="pl-en">A</span>] <span class="pl-k">=</span> <span class="pl-en">Fix</span>[<span class="pl-en">BinaryTree</span>[<span class="pl-en">A</span>, <span class="pl-k">?</span>]]
<span class="pl-k">type</span> <span class="pl-en">RecursiveJson</span> <span class="pl-k">=</span> <span class="pl-en">Fix</span>[<span class="pl-en">Json</span>]</pre></div>
<p>...which is a bit long-winded and unpleasant.
Let's rename things.</p>
<p>The most-common convention I've seen is to append an <code>F</code> for "functor" to the
data type and use the proper name in the alias.</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">IntListF</span>[<span class="pl-k">+</span><span class="pl-en">F</span>]
<span class="pl-k">type</span> <span class="pl-en">IntList</span> <span class="pl-k">=</span> <span class="pl-en">Fix</span>[<span class="pl-en">IntListF</span>]
<span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">BinaryTreeF</span>[<span class="pl-k">+</span><span class="pl-en">A</span>, <span class="pl-k">+</span><span class="pl-en">F</span>]
<span class="pl-k">type</span> <span class="pl-en">BinaryTree</span>[<span class="pl-en">A</span>] <span class="pl-k">=</span> <span class="pl-en">Fix</span>[<span class="pl-en">BinaryTreeF</span>[<span class="pl-en">A</span>, <span class="pl-k">?</span>]]
<span class="pl-k">sealed</span> <span class="pl-k">trait</span> <span class="pl-en">JsonF</span>[<span class="pl-k">+</span><span class="pl-en">F</span>]
<span class="pl-k">type</span> <span class="pl-en">Json</span> <span class="pl-k">=</span> <span class="pl-en">Fix</span>[<span class="pl-en">JsonF</span>]</pre></div>
<p>This actually works out great because then, in practice, I often create a new
object with helpers that improve ergonomics and avoid <code>Fix</code> boilerplate when you
need to manually create a structure, for example, unit test expectations,
or using a parser that doesn't play nice with recursion schemes.</p>
<p>For example:</p>
<div class="highlight highlight-source-scala"><pre><span class="pl-k">type</span> <span class="pl-en">IntList</span> <span class="pl-k">=</span> <span class="pl-en">Fix</span>[<span class="pl-en">IntListF</span>]
<span class="pl-k">object</span> <span class="pl-en">IntList</span> {
<span class="pl-c"><span class="pl-c">//</span> Helpful cos Scala's type inference fails so often</span>
<span class="pl-k">def</span> <span class="pl-en">apply</span>(<span class="pl-v">f</span>: <span class="pl-en">IntListF</span>[<span class="pl-en">IntList</span>])<span class="pl-k">:</span> <span class="pl-en">IntList</span> <span class="pl-k">=</span>
<span class="pl-en">Fix</span>(f)
<span class="pl-k">def</span> <span class="pl-en">nil</span><span class="pl-k">:</span> <span class="pl-en">IntList</span> <span class="pl-k">=</span>
apply(<span class="pl-en">IntNil</span>)
<span class="pl-k">def</span> <span class="pl-en">cons</span>(<span class="pl-v">head</span>: <span class="pl-k">Int</span>, <span class="pl-v">tail</span>: <span class="pl-en">IntList</span>)<span class="pl-k">:</span> <span class="pl-en">IntList</span> <span class="pl-k">=</span>
apply(<span class="pl-en">IntCons</span>(head, tail))
<span class="pl-k">def</span> <span class="pl-en">fromList</span>(<span class="pl-v">is</span>: <span class="pl-k">Int</span><span class="pl-k">*</span>)<span class="pl-k">:</span> <span class="pl-en">IntList</span> <span class="pl-k">=</span>
is.foldRight(nil)(cons)
}</pre></div>
<h1><a href="#done" aria-hidden="true" class="anchor" id="user-content-done"><svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Done!</h1>
<p>That's it! We're done.
It may seem like a lot of work but there's benefit coming that greatly outweighs
the cost. The cost isn't huge anyway, just a bit of one-time-only boilerplate.</p>
<p>All source code available here:
<a href="https://github.com/japgolly/learning/tree/public/recursion-blog/example/src/main/scala/japgolly/blog/recursion">https://github.com/japgolly/learning/tree/public/recursion-blog/example/src/main/scala/japgolly/blog/recursion</a></p>
<p>If you like what I do and you'd like to support me, this series or any of my other projects, <a href="https://www.patreon.com/japgolly">become a patron!</a> It will make a big difference to me and help further more content. You can also <a href="https://twitter.com/japgolly">follow me on Twitter</a> for updates about the next post the series.</p>
</article>
</div>
</div>
</div>
</div>
</div>
</div>
David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com5tag:blogger.com,1999:blog-5837865295676240265.post-18048992725601542032017-06-05T21:27:00.002+10:002017-06-05T21:27:31.517+10:00Dependently-Typed Functions<div class="style2017">
<p>It's been a while since my last blog post.
This time around I'm going to show how to do something in Scala that you might first
think would be very straight-forward but unfortunately, at least in Scala ≤ 2.12.2, requires a bit of hoop-jumping:
dependently-typed functions.</p>
<p>Consider the following data type:</p>
<pre><code class="language-scala"><span class="hljs-keyword">sealed</span> <span class="hljs-class"><span class="hljs-keyword">trait</span> <span class="hljs-title">Field</span> </span>{ <span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">Value</span> }</span>
<span class="hljs-class"><span class="hljs-keyword">object</span> <span class="hljs-title">Field</span> </span>{
<span class="hljs-keyword">case</span> <span class="hljs-class"><span class="hljs-keyword">object</span> <span class="hljs-title">Name</span> <span class="hljs-keyword">extends</span> <span class="hljs-title">Field</span> </span>{ <span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">Value</span> </span>= <span class="hljs-type">String</span> }
<span class="hljs-keyword">case</span> <span class="hljs-class"><span class="hljs-keyword">object</span> <span class="hljs-title">Age</span> <span class="hljs-keyword">extends</span> <span class="hljs-title">Field</span> </span>{ <span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">Value</span> </span>= <span class="hljs-type">Int</span> }
}
</code></pre>
<p>With this data type, you have the following path-dependent types:</p>
<pre style="width:100%; border:"><code class="language-scala"><span class="hljs-type">Name</span>.<span class="hljs-type">Value</span> == <span class="hljs-type">String</span>
<span class="hljs-type">Age</span> .<span class="hljs-type">Value</span> == <span class="hljs-type">Int</span>
<span class="hljs-keyword">val</span> n = <span class="hljs-type">Name</span>
n.<span class="hljs-type">Value</span> == <span class="hljs-type">String</span>
</code></pre>
<p>You also have the following type projections:</p>
<pre><code class="language-scala"><span class="hljs-type">Name</span> .<span class="hljs-keyword">type</span>#<span class="hljs-type">Value</span> = <span class="hljs-type">String</span>
<span class="hljs-type">Age</span> .<span class="hljs-keyword">type</span>#<span class="hljs-type">Value</span> = <span class="hljs-type">Int</span>
<span class="hljs-type">Field</span> #<span class="hljs-type">Value</span> = <span class="hljs-type">Any</span>
</code></pre>
<h1>1. A Dependently-Typed Method</h1>
<p>Lets say you want to create the following, rather innocent-looking function:</p>
<pre><code class="language-scala"><span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">emptyValue</span></span>(f: <span class="hljs-type">Field</span>): f.<span class="hljs-type">Value</span>
</code></pre>
<p>Likely your first attempt will be to pattern-match:</p>
<pre><code class="language-scala"><span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">emptyValue</span></span>(f: <span class="hljs-type">Field</span>): f.<span class="hljs-type">Value</span> =
f <span class="hljs-keyword">match</span> {
<span class="hljs-keyword">case</span> <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span> => <span class="hljs-string">""</span>
<span class="hljs-keyword">case</span> <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span> => <span class="hljs-number">0</span>
}
</code></pre>
<p>Unfortunately scalac doesn't support this and you instead get compilation errors:</p>
<pre><code><console>:15: error:<span class="hljs-built_in"> type </span>mismatch;
found : String(<span class="hljs-string">""</span>)
required: f.Value
case Field.Name => <span class="hljs-string">""</span>
^
<console>:16: error:<span class="hljs-built_in"> type </span>mismatch;
found : Int(0)
required: f.Value
case Field.Age => 0
^
</code></pre>
<p>Ok, let's switch to a different representation:</p>
<pre><code class="language-scala"><span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">Aux</span>[<span class="hljs-type">A</span>] </span>= <span class="hljs-type">Field</span> { <span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">Value</span> </span>= <span class="hljs-type">A</span> }
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">emptyValueHack</span></span>[<span class="hljs-type">A</span>](f: <span class="hljs-type">Aux</span>[<span class="hljs-type">A</span>]): <span class="hljs-type">A</span> =
f <span class="hljs-keyword">match</span> {
<span class="hljs-keyword">case</span> <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span> => <span class="hljs-string">""</span>
<span class="hljs-keyword">case</span> <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span> => <span class="hljs-number">0</span>
}
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">emptyValue</span></span>(f: <span class="hljs-type">Field</span>): f.<span class="hljs-type">Value</span> =
emptyValueHack[f.<span class="hljs-type">Value</span>](f)
</code></pre>
<p>This <em>does</em> work. Great! END BLOG POST. Right?
Wrong. There's another problem.
What happens if we forget a case:</p>
<pre><code class="language-scala"><span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">emptyValueHack2</span></span>[<span class="hljs-type">A</span>](f: <span class="hljs-type">Aux</span>[<span class="hljs-type">A</span>]): <span class="hljs-type">A</span> =
f <span class="hljs-keyword">match</span> {
<span class="hljs-keyword">case</span> <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span> => <span class="hljs-string">""</span>
<span class="hljs-comment">// case Field.Age => 0</span>
}
</code></pre>
<p>It's compiles successfully without any warnings.
We've lost exhaustivity checking which is a runtime exception just waiting to happen.
Oh noes. Now what?
Consider: what are we doing when we do simple pattern-matching on each case?
We're inspecting a value, choosing a corresponding function, and executing it.
If each case has exactly one corresponding function then we have exhaustivity.
That sounds like something we already have the means to do, without pattern-matching, in plain old Scala.</p>
<p>Ok, let do this ourselves, exactly one function per case and choose depending on the <code>Field</code>.</p>
<pre><code class="language-scala"><span class="hljs-keyword">sealed</span> <span class="hljs-class"><span class="hljs-keyword">trait</span> <span class="hljs-title">Field</span> </span>{
<span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">Value</span></span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">fold</span></span>(n: <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.<span class="hljs-keyword">type</span> => <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.<span class="hljs-type">Value</span>,
a: <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span> .<span class="hljs-keyword">type</span> => <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span> .<span class="hljs-type">Value</span>): <span class="hljs-type">Value</span>
}
<span class="hljs-class"><span class="hljs-keyword">object</span> <span class="hljs-title">Field</span> </span>{
<span class="hljs-keyword">case</span> <span class="hljs-class"><span class="hljs-keyword">object</span> <span class="hljs-title">Name</span> <span class="hljs-keyword">extends</span> <span class="hljs-title">Field</span> </span>{
<span class="hljs-keyword">override</span> <span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">Value</span> </span>= <span class="hljs-type">String</span>
<span class="hljs-keyword">override</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">fold</span></span>(n: <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.<span class="hljs-keyword">type</span> => <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.<span class="hljs-type">Value</span>, a: <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span>.<span class="hljs-keyword">type</span> => <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span>.<span class="hljs-type">Value</span>): <span class="hljs-type">Value</span> =
n(<span class="hljs-keyword">this</span>)
}
<span class="hljs-keyword">case</span> <span class="hljs-class"><span class="hljs-keyword">object</span> <span class="hljs-title">Age</span> <span class="hljs-keyword">extends</span> <span class="hljs-title">Field</span> </span>{
<span class="hljs-keyword">override</span> <span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">Value</span> </span>= <span class="hljs-type">Int</span>
<span class="hljs-keyword">override</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">fold</span></span>(n: <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.<span class="hljs-keyword">type</span> => <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.<span class="hljs-type">Value</span>, a: <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span>.<span class="hljs-keyword">type</span> => <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span>.<span class="hljs-type">Value</span>): <span class="hljs-type">Value</span> =
a(<span class="hljs-keyword">this</span>)
}
}
</code></pre>
<p>Take a good look at this. There are a few things that might seem odd.
You're probably wondering why I'm passing statically-known singletons to arguments.
Why not just:</p>
<pre><code class="language-scala"><span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">fold</span></span>(name: => <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.<span class="hljs-type">Value</span>,
age : => <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span> .<span class="hljs-type">Value</span>): <span class="hljs-type">Value</span>
</code></pre>
<p>Three reasons:</p>
<ol>
<li><p>Cases aren't always objects. What if they're case classes like <code>case class CustomField(label: String) extends Field</code>. In such a case it will be important to pass the instance to the caller so that they have access to the additional/dynamic information in the <code>label</code> field.</p></li>
<li><p>It embodies the proof that the appropriate argument case is required for each field case. The types make it clear and so long as you call the arg with <code>this</code> instead of <code>Field.Name</code> directly, then you get an extra bit of proof of correctness. Seeing as the workarounds described in this post introduce some tedium so too do they often introduce copy-pasting which can lead to accidental bugs like this:</p></li>
</ol>
<pre><code class="language-scala"><span class="hljs-comment">// Spot the bug...</span>
<span class="hljs-keyword">case</span> <span class="hljs-class"><span class="hljs-keyword">object</span> <span class="hljs-title">Name</span> <span class="hljs-keyword">extends</span> <span class="hljs-title">Field</span> </span>{
<span class="hljs-keyword">override</span> <span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">Value</span> </span>= <span class="hljs-type">String</span>
<span class="hljs-keyword">override</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">fold</span></span>(n: <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.<span class="hljs-keyword">type</span> => <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.<span class="hljs-type">Value</span>, a: <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span>.<span class="hljs-keyword">type</span> => <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span>.<span class="hljs-type">Value</span>): <span class="hljs-type">Value</span> =
n(<span class="hljs-type">Name</span>)
}
<span class="hljs-keyword">case</span> <span class="hljs-class"><span class="hljs-keyword">object</span> <span class="hljs-title">Age</span> <span class="hljs-keyword">extends</span> <span class="hljs-title">Field</span> </span>{
<span class="hljs-keyword">override</span> <span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">Value</span> </span>= <span class="hljs-type">Int</span>
<span class="hljs-keyword">override</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">fold</span></span>(n: <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.<span class="hljs-keyword">type</span> => <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.<span class="hljs-type">Value</span>, a: <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span>.<span class="hljs-keyword">type</span> => <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span>.<span class="hljs-type">Value</span>): <span class="hljs-type">Value</span> =
n(<span class="hljs-type">Name</span>)
}
</code></pre>
<ol start="3">
<li>The caller has the same problem as above; if they're pattern-matching and want an downcast instance of <code>Field</code> in their case functions then this gives it to them so that <em>they</em> don't manually reference <code>Field.Name</code> and end up with potential bugs.</li>
</ol>
<h1>2. Reducing Duplication</h1>
<p>Next, let's think about what happens if we add a new case like <code>Field.Address</code>; we'll have to update the <code>fold</code> signature in our current three places and then add a fourth in <code>Field.Address</code>. So much repetition!
If you find yourself copy-pasting a cumbersome list of method arguments, consider it a good practice to encapsulate them all in a class/record.
That way you can consistently pass around and declare a single type, and changes to its fields don't cause changes to method signatures.
Let's do that for our fold method:</p>
<pre><code class="language-scala"><span class="hljs-keyword">sealed</span> <span class="hljs-class"><span class="hljs-keyword">trait</span> <span class="hljs-title">Field</span> </span>{
<span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">Value</span></span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">fold</span></span>(f: <span class="hljs-type">Field</span>.<span class="hljs-type">Fold</span>): <span class="hljs-type">Value</span>
}
<span class="hljs-class"><span class="hljs-keyword">object</span> <span class="hljs-title">Field</span> </span>{
<span class="hljs-keyword">case</span> <span class="hljs-class"><span class="hljs-keyword">object</span> <span class="hljs-title">Name</span> <span class="hljs-keyword">extends</span> <span class="hljs-title">Field</span> </span>{
<span class="hljs-keyword">override</span> <span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">Value</span> </span>= <span class="hljs-type">String</span>
<span class="hljs-keyword">override</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">fold</span></span>(f: <span class="hljs-type">Field</span>.<span class="hljs-type">Fold</span>) = f.name(<span class="hljs-keyword">this</span>)
}
<span class="hljs-keyword">case</span> <span class="hljs-class"><span class="hljs-keyword">object</span> <span class="hljs-title">Age</span> <span class="hljs-keyword">extends</span> <span class="hljs-title">Field</span> </span>{
<span class="hljs-keyword">override</span> <span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">Value</span> </span>= <span class="hljs-type">Int</span>
<span class="hljs-keyword">override</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">fold</span></span>(f: <span class="hljs-type">Field</span>.<span class="hljs-type">Fold</span>) = f.age(<span class="hljs-keyword">this</span>)
}
<span class="hljs-keyword">final</span> <span class="hljs-keyword">case</span> <span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">Fold</span>(<span class="hljs-params">name: <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.type => <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.<span class="hljs-type">Value</span>,
age : <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span> .type => <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span> .<span class="hljs-type">Value</span></span>)</span>
}
</code></pre>
<p>Look at that <code>fold</code> method, nice and easy. We can add cases ALL DAY LONG without repetition.
High five thineself.</p>
<p>While we're in the vicinity, let's add a little convenience method to the <code>Fold</code> class to really prove to ourselves that can accomplish our original goal, pay attention to signature:</p>
<pre><code class="language-scala"><span class="hljs-keyword">final</span> <span class="hljs-keyword">case</span> <span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">Fold</span>(<span class="hljs-params">name: <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.type => <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.<span class="hljs-type">Value</span>,
age : <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span> .type => <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span> .<span class="hljs-type">Value</span></span>) </span>{
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">apply</span></span>(f: <span class="hljs-type">Field</span>): f.<span class="hljs-type">Value</span> =
f.fold(<span class="hljs-keyword">this</span>)
}
</code></pre>
<p>Now using the above, how can we rewrite our original <code>emptyValue</code> function?</p>
<pre><code class="language-scala"><span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">emptyValue</span></span>(f: <span class="hljs-type">Field</span>): f.<span class="hljs-type">Value</span> =
<span class="hljs-type">Field</span>.<span class="hljs-type">Fold</span>(
name = _ => <span class="hljs-string">""</span>,
age = _ => <span class="hljs-number">0</span>,
)(f)
</code></pre>
<p>Goals: accomplished.</p>
<ul>
<li>Concise</li>
<li>Exhaustive</li>
<li>Pattern-match in spirit</li>
</ul>
<p>It's a little ugly but not a huge deal.
Depending on your codebase and environment you can likely avoid the need for <code>_ =></code> bits without penalty which will make it a little nicer on the eyes.</p>
<h1>3. A Dependently-Typed Function</h1>
<p>Like most everything from OOP, methods are boring. You can't abstract over them,
all you can do is call them. Functions allow you to do more because they themselves
are values; you can pass them around which facilitates awesome collections methods
like <code>.map(A => B)</code>, <code>.filter(A => Boolean)</code>, etc. and that's just the tip of the iceberg.</p>
<p>In the previous step we created a method whose output depends on its input <em>value</em>.
How can we have the same in the form of a function?
Function types are fixed, and the function type doesn't have a value to work with.
Scala doesn't accept this kind of syntax:</p>
<pre><code class="language-scala"><span class="hljs-comment">// nope - not valid syntax</span>
<span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">MyFn</span> </span>= (f: <span class="hljs-type">Field</span>) => f.<span class="hljs-type">Value</span>
</code></pre>
<p>You might try using projections like this but you'll lose the relationship between input and output.</p>
<pre><code class="language-scala"><span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">MyFn</span> </span>= <span class="hljs-type">Field</span> => <span class="hljs-type">Field</span>#<span class="hljs-type">Value</span>
<span class="hljs-comment">// which is the same as</span>
<span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">MyFn</span> </span>= <span class="hljs-type">Field</span> => <span class="hljs-type">Any</span>
</code></pre>
<p>We've already encountered the answer. Surprise!
<code>Field.Fold</code> is what we're looking for. It's a value, you can pass it around;
you can apply it via its <code>apply</code> method to obtain a dependently-typed result.</p>
<p>Let's try it out:</p>
<pre><code class="language-scala"><span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">blah</span></span>(field: <span class="hljs-type">Field</span>, getValue: <span class="hljs-type">Field</span>.<span class="hljs-type">Fold</span>): field.<span class="hljs-type">Value</span> =
getValue(field)
</code></pre>
<p>Hmmm, yes, well it <em>does</em> work but it's admittedly not very interesting in that shape.
It can only represent <code>f => f.Value</code>...</p>
<h1>4. Dependently-Typed Functions</h1>
<p>What if we want a function of any shape instead of just <code>f => f.Value</code>?
Maybe you want to be able to print each field to the screen: <code>f => f.Value => Unit</code>.
Maybe you want to reduce lists of each value to a single value: <code>f => List[f.Value] => f.Value</code>.
Or perform validation: <code>f => f.Value => Either[Error, f.Value]</code>.</p>
<p>Let's find an abstraction that can represent each example above.
They all have the shape: <code>f => {something including f.Value}</code>.
Add some type aliases for the right-hand sides:</p>
<pre><code class="language-scala"><span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">GetValue</span> [<span class="hljs-type">Value</span>] </span>= <span class="hljs-type">Value</span>
<span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">Print</span> [<span class="hljs-type">Value</span>] </span>= <span class="hljs-type">Value</span> => <span class="hljs-type">Unit</span>
<span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">ReduceList</span>[<span class="hljs-type">Value</span>] </span>= <span class="hljs-type">List</span>[<span class="hljs-type">Value</span>] => <span class="hljs-type">Value</span>
<span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">Validate</span> [<span class="hljs-type">Value</span>] </span>= <span class="hljs-type">Value</span> => <span class="hljs-type">Either</span>[<span class="hljs-type">Error</span>, <span class="hljs-type">Value</span>]
</code></pre>
<p>There's our abstraction:</p>
<pre><code class="language-scala"><span class="hljs-class"><span class="hljs-keyword">type</span> <span class="hljs-title">F</span>[<span class="hljs-type">Value</span>] </span>= <span class="hljs-comment">// various</span>
</code></pre>
<p>with the fold being various cases of <code>f => F[f.Value] where F[Value] = …</code>.</p>
<p>Let's update the fold to allow this:</p>
<pre><code class="language-scala"><span class="hljs-keyword">final</span> <span class="hljs-keyword">case</span> <span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">Fold</span>[<span class="hljs-type">F</span>[_]](<span class="hljs-params">name: <span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.type => <span class="hljs-type">F</span>[<span class="hljs-type">Field</span>.<span class="hljs-type">Name</span>.<span class="hljs-type">Value</span>],
age : <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span> .type => <span class="hljs-type">F</span>[<span class="hljs-type">Field</span>.<span class="hljs-type">Age</span> .<span class="hljs-type">Value</span>]</span>) </span>{
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">apply</span></span>(f: <span class="hljs-type">Field</span>): <span class="hljs-type">F</span>[f.<span class="hljs-type">Value</span>] =
f.fold(<span class="hljs-keyword">this</span>)
}
</code></pre>
<p>Now let's try it out:</p>
<pre><code class="language-scala"><span class="hljs-comment">// f: Field => f.Value</span>
<span class="hljs-keyword">val</span> getValue = <span class="hljs-type">Field</span>.<span class="hljs-type">Fold</span>[<span class="hljs-type">GetValue</span>](
name = _ => <span class="hljs-string">"George"</span>,
age = _ => <span class="hljs-number">99</span>)
<span class="hljs-comment">// f: Field => f.Value => Unit</span>
<span class="hljs-keyword">val</span> printValue = <span class="hljs-type">Field</span>.<span class="hljs-type">Fold</span>[<span class="hljs-type">Print</span>](
name = _ => n => println(<span class="hljs-string">s"My name is <span class="hljs-subst">$n</span>."</span>),
age = _ => i => println(<span class="hljs-string">s"I am <span class="hljs-subst">$i</span> years old."</span>))
</code></pre>
<p>Great. Dependently-typed function values. And how do we use them? Just like normal functions.</p>
<pre><code class="language-scala"><span class="hljs-keyword">val</span> f = <span class="hljs-type">Field</span>.<span class="hljs-type">Age</span>
<span class="hljs-keyword">val</span> v = getValue(f) <span class="hljs-comment">// v = 99</span>
printValue(f)(v) <span class="hljs-comment">// prints: I am 99 years old.</span>
</code></pre>
<p>Very good. I hope you found the interesting. Thanks for reading!</p>
<p>For more exhilaration, as an exercise to the reader, try to:</p>
<ul>
<li>define composition of fold functions</li>
<li>create multiple folds for subsets of the same data type</li>
</ul>
<h1>Appendix</h1>
<h3>Type parameters</h3>
<p>In terms of representation, using type parameters is isomorphic (ignoring variance).
The following represents the exact same information:</p>
<pre><code class="language-scala"><span class="hljs-keyword">sealed</span> <span class="hljs-class"><span class="hljs-keyword">trait</span> <span class="hljs-title">Field</span>[<span class="hljs-type">V</span>]</span>
<span class="hljs-class"><span class="hljs-keyword">object</span> <span class="hljs-title">Field</span> </span>{
<span class="hljs-keyword">case</span> <span class="hljs-class"><span class="hljs-keyword">object</span> <span class="hljs-title">Name</span> <span class="hljs-keyword">extends</span> <span class="hljs-title">Field</span>[<span class="hljs-type">Int</span>]</span>
<span class="hljs-keyword">case</span> <span class="hljs-class"><span class="hljs-keyword">object</span> <span class="hljs-title">Age</span> <span class="hljs-keyword">extends</span> <span class="hljs-title">Field</span>[<span class="hljs-type">String</span>]</span>
}
</code></pre>
<p>In terms of usage however, the two representations are far from the same and have a drastic impact on how you'll use them.
Some things are easier, some harder.</p>
<p>For example, in terms of generic access existential types make life easier: its simply <code>Field</code> instead of <code>Field[_]</code>. This is even more evident when there's a constraint on the type like <code>Field[_ <: AnyRef]</code> whilst <code>Field</code> remains the same.</p>
<p>I've come across other scenarios but honestly can't remember right now, something something implicits...
Instead take a look at this slide deck of <a href="https://twitter.com/julienrf">@julienrf</a>'s for more comparison: <a href="https://julienrf.github.io/2017/existential-types/">https://julienrf.github.io/2017/existential-types/</a></p>
<h3>Code</h3>
<p>The final code in full of this blog post is available here:
<a href="https://gist.github.com/japgolly/162f102d516abf86b54359ef0b1d3b65">https://gist.github.com/japgolly/162f102d516abf86b54359ef0b1d3b65</a></p>
</div>David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com0tag:blogger.com,1999:blog-5837865295676240265.post-13712180152357551932016-03-20T16:04:00.001+11:002016-03-21T11:23:24.356+11:00Testing Scala.JS on Firefox & Chrome from SBT<div dir="ltr" style="text-align: left; margin-bottom:3em" trbidi="on">
It's been fantastic being able to write Scala and compile it to JavaScript thanks to <a href="http://www.scala-js.org/">Scala.JS</a>. Going further, Scala.JS lets you write accompanying unit tests, run them from SBT, and choose a target environment from {Rhino, Node.JS, Phantom.JS}. If you needed DOM in your testing, your only option was Phantom.JS which seems great at first but—because it only simulates DOM—you soon discover that there are many cases in which its behaviour diverges from normal browsers (eg. DOM types for <code><td></code> tags), or isn't supported at all (text selection, anything to do with focus, more). Oh, and it's also riddled with bugs, many significant and long-standing. So while Phantom.JS's effort is appreciated, it's no substitute for a real browser. This was where the story ended until very recently.<br />
<br />
Recently, the Scala.JS team released <a href="https://github.com/scala-js/scala-js-env-selenium">scala-js-env-selenium</a> which allows you to use <a href="http://www.seleniumhq.org/">Selenium</a> in the same way you would the other JS environments. That means that Scala.JS can now interface with <em>real</em> browsers, namely Firefox and Chrome. Awesome!<br />
<br>
The next step, and the purpose of this blog post, is to effectively integrate it into your SBT/Scala/Scala.JS project.
Let me tell you about my ideal environment and then I'll show you how to achieve it.
<h1>Goals</h1>
In my ideal environment:<br />
<ul style="text-align: left;">
<li>I write my Scala.JS unit tests the same way I already do, and I can continue to run them against Node.JS or Phantom.JS because it's fast. No changes there.</li>
<li>To run tests against Firefox or Chrome, I simply prefix my SBT command with <code>firefox:</code> or <code>chrome:</code>.<br>For example, <code>firefox:test</code> would run tests in Firefox, <code>chrome:testOnly a.b.c</code> would run my <code>a.b.c</code> test in Chrome.</li>
<li>Testing in FF/Chrome doesn't require recompilation <em>(All in the Land know Scalac is slow)</em>. It should use all of the same bits and bobs (especially the output JS) that normal tests use.</li>
<li>I specify that certain tests will only run in FF and/or Chrome.<br>For example, tests that use focus should skip Phantom.JS where I know focus doesn't work.</li>
<li>I run <code>testAll</code> to run the same tests in all environments (fast-env & Firefox & Chrome). This will happen concurrently.</li>
<li>I can use FF/Chrome headlessly (i.e. without the windows popping up when the browsers are launched and running).</li>
</ul>
<h1>1. Selenium support.</h1>
Add this to your <code>project/plugins.sbt</code>:
<pre class="brush: scala">
libraryDependencies += "org.scala-js" %% "scalajs-env-selenium" % "0.1.1"
</pre>
Then install <a href="https://sites.google.com/a/chromium.org/chromedriver/">ChromeDriver</a> which is needed so that Selenium can interface with Chrome.
<h1>2. General SBT Config</h1>
Create a file called <code>project/InBrowserTesting.scala</code> with the following content:
<script src="https://gist.github.com/japgolly/5e29d4bae9141df5e053.js"></script>
This creates SBT configurations for each browser, then delegates most of the settings to the <code>test:*</code> settings.
<h1>3. Project-specific SBT config</h1>
Next you need to look at your existing SBT build settings.<br /><br>
For each cross-JVM/JS project, add:<br />
<pre class="brush: scala">
.configure(InBrowserTesting.cross)
</pre>
For each JS-only project, add:<br />
<pre class="brush: scala">
.configure(InBrowserTesting.js)
</pre>
For each JVM-only project, add:<br />
<pre class="brush: scala">
.configure(InBrowserTesting.jvm)
</pre>
<br>
You might be wondering why JVM projects need any configuration at all. It's so that when <code>testAll</code> is run from a JVM project or the root project, you want it to run the JVM tests. Without this setting, <code>testAll</code> in a JVM project would do nothing at all, and <code>testAll</code> from the root would only run the JS tests.
<h1>4. Environment-Dependent Tests</h1>
I often forget that I'm writing JS when I write Scala.JS. As we're in JS land, to determine our environment all we have to do is check the user-agent.
<script src="https://gist.github.com/japgolly/9818f557fc66384b7eab.js"></script>
How you skip tests depends on the test framework you're using but all you have to do is put something like this in your test code:<br />
<pre class="brush: scala">
if (JsEnvUtils.isRealBrowser) {
// test here
} else {
// skip
}
</pre>
<h1>5. Headlessness</h1>
Super simple (unless you're a Windows user). Install xvfb, "X Virtual FrameBuffer", which starts X without a graphics display.<br><br>
Here are two different ways you can use it:
<ol>
<li>
Either start it in a separate window via<br />
<pre style="background:#eee">Xvfb :1
</pre>
or in the background<br />
<pre style="background:#eee">nohup Xvfb :1 &
</pre>
then launch SBT like this:<br />
<pre style="background:#eee">DISPLAY=:1 sbt
</pre>
</li>
<li style="margin-top:1em">This tip comes from danielkza <em>(thanks!)</em>.
You can simply prepend your SBT command with <code>xvfb-run -a</code> to have an X server spun up on demand without the need to start it yourself. Beware though, that <code>xvfb-run</code> is <a href="https://github.com/revnode/xvfb-run/blob/master/xvfb-run#L85-L97">a bit naive</a> and <a href="http://stackoverflow.com/questions/30332137/xvfb-run-unreliable-when-multiple-instances-invoked-in-parallel">susceptible to race conditions</a> so while it'll be fine on your local machine, it may cause you problems on your CI server or similar.
</li>
</ol>
You'll no longer see any Firefox and Chrome windows; all your output just appears in the SBT console as usual. Too easy.
<p><em>Note: The <code>:1</code> indicates the X display-number, which is a means to uniquely identify an X server on a host. The <code>1</code> is completely arbitrary—you can choose any number so long as there isn't another X server running that's already associated with it.</em></p>
<h1>Done.</h1>
All goals described are thus achieved.<br>
I hope you found this helpful.<br>
Happy coding!</div>David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com4tag:blogger.com,1999:blog-5837865295676240265.post-68630553948822580582015-02-23T19:18:00.000+11:002015-02-23T19:18:14.249+11:00Zero-overhead Recursive ADT Coproducts<p><em>Zero-product Recursive AMD what???</em></p>
<p>Ok. Imagine this:
you're building some app, and in certain parts users can type text
with special tokens. (It's like in Twitter, when typing “<code>Hello @japgolly</code>”
the “<code>@japgolly</code>” part gets special treatment.) You parse various
types of tokens. You might have different locations to type text, and
rules about which tokens are allowed in each location. You want the
compiler to enforce those rules but you also want to handle
tokens generically sometimes. How would you do such a thing in
Scala?</p>
<h1>Initial Attempts</h1>
<p>
Ideally you'd define an ADT (<a href="http://en.wikipedia.org/wiki/Algebraic_data_type">algebraic data type</a>) for all tokens possible, then create new
types for each location that form a subset of tokens allowed. If
that were possible, here's what that would look like.
</p>
<pre class="brush: scala">
sealed trait Token
case class PlainText(t: String) extends Token
case object NewLine extends Token
case class Link(url: String) extends Token
case class Quote(q: List[Token]) extends Token
// type BlogTitle = PlainText | Link | Quote[BlogTitle]
// type BlogComment = PlainText | NewLine | Quote[BlogComment]
</pre>
<p>
Now Scala won't let
us create our <code>BlogTitle</code> type like shown above. It doesn't have a
syntax for <a href="http://en.wikipedia.org/wiki/Coproduct">coproducts</a> (which is what <code>BlogTitle</code> and <code>BlogComment</code> would
be, also called “disjoint unions” and “sum-types”) over
existing types. Seeing as we have control over the definition of the
generic tokens, we can be tricky and inverse the declarations like
this:
</p>
<pre class="brush: scala">
sealed trait BlogTitle
sealed trait BlogComment
sealed trait Token
case class PlainText(t: String) extends Token with BlogTitle with BlogComment
case object NewLine extends Token with BlogComment
case class Link(url: String) extends Token with BlogTitle
case class Quote(q: List[????]) extends Token with BlogTitle with BlogComment
// ↑ hmmm...
</pre>
<p>
...but as you can
see, we hit a wall when we to <code>Quote</code>, which is recursive. We want a
<code>Quote</code> in a <code>BlogTitle</code> to only contain <code>BlogTitle</code> tokens, not just any
type of token. We can continue our poor hack as follows.
</p>
<pre class="brush: scala">
abstract class Quote[A <: Token] extends Token { val q: List[A] }
case class QuoteInBlogTitle (q: List[BlogTitle]) extends Quote[BlogTitle]
case class QuoteInBlogComment(q: List[BlogComment]) extends Quote[BlogComment]
</pre>
<p>
Not pleasant. And we're not really sharing types anymore. What else could we do?
</p>
<p>
We could create
separate ADTs for <code>BlogTitle</code> and <code>BlogComment</code>, that would mirror and
wrap their matching generic tokens, then write converters from
specific to generic. That's a lot of duplicated logic and tedium,
plus we now double the allocations and memory needed. Let's try something else...
</p>
<h1>Shapeless</h1>
<p><em>NOTE: This bit about Shapeless is an interesting detour, but it can be skipped.</em></p>
<p>
We could use <a href="https://github.com/milessabin/shapeless">Shapeless</a>!
Shapeless is an ingeniously-sculpted library that
facilitates abstractions that a panel of sane experts would deem
impossible in Scala, one such abstraction being <code>Coproduct</code>s. Here's what
a solution looks like using Shapeless.</p>
<p><em>(Sorry I thought I had this working but just realised recursive coproducts don't work. I've commented-out Quote[A] for now. There is probably a way of doing this – Shapeless often doesn't take no from scalac for an answer – I'll update this if some kind 'netizen shares how.)</em>
</p>
<pre class="brush: scala">
sealed trait Token
case class PlainText(text: String) extends Token
case object NewLine extends Token
case class Link(url: String) extends Token
case class Quote[A](q: List[A]) extends Token
type BlogTitle = PlainText :+: Link :+: /*Quote[BlogTitle] :+: */ CNil
type BlogComment = PlainText :+: NewLine.type :+: /*Quote[BlogComment] :+: */ CNil
/* compiles → */ val title = Coproduct[BlogTitle](PlainText("cool"))
// error → // val title = Coproduct[BlogTitle](NewLine)
</pre>
<p>
So far so good. What would a <code style="background:#eee">Token ⇒ Text</code> function look like?
</p>
<pre class="brush: scala">
object ToText extends Poly1 {
implicit def caseText = at[PlainText ](_.text)
implicit def caseNewLine = at[NewLine.type](_ => "\n")
implicit def caseLink = at[Link ](_.url)
// ...
}
val text: String = title.map(ToText).unify
</pre>
<p>
Ok I'm a little unhappy because I'm very fold of
pattern-matching in these situations, but the above <i>does</i> work
effectively. One thing to be aware of with Shapeless, is <i>how</i>
it works. To achieve its awesomeness, it must build up a hierarchy of
proofs which incurs time and space costs at both compile- and
run-time – its awesomeness ain't free. The <code style="background:#eee">val title = ...</code>
statement above creates at least 7 new classes at runtime, where I want 1.
Depending on your usage and needs, that overhead might be nothing,
but it might be significant. It's something to be aware of when you
decide on your solution.
</p>
<h1>Zero-overhead Recursive ADT Coproducts</h1>
<p>
There's another way.
I mentioned “zero-overhead” and it can be done. Here is a
different solution that relies solely on standard Scala features, one
such feature being path-dependent types.
</p>
<p>
You can create an
abstract ADT, putting each constituent in a trait, then simply
combine those traits into an object to have it reify a new, concrete,
sealed ADT. <b>Sealed!</b> Let's see the new definition:
</p>
<pre class="brush: scala">
// Generic
sealed trait Base {
sealed trait Token
}
sealed trait PlainTextT extends Base {
case class PlainText(text: String) extends Token
}
sealed trait NewLineT extends Base {
case class NewLine() extends Token
}
sealed trait LinkT extends Base {
case class Link(url: String) extends Token
}
sealed trait QuoteT extends Base {
case class Quote(content: List[Token]) extends Token
}
// Specific
object BlogTitle extends PlainTextT with LinkT with QuoteT
object BlogComment extends PlainTextT with NewLineT with QuoteT
</pre>
<p>Now let's use it:</p>
<pre class="brush: scala">
List[BlogTitle.Token](BlogTitle.PlainText("Hello")) // success
// List[BlogTitle.Token](BlogTitle.NewLine) ← error: BlogTitle.NewLine doesn't exist
// List[BlogTitle.Token](BlogComment.NewLine) ← error: BlogComment tokens aren't BlogTitle tokens
// Specific
val blogTitleToText: BlogTitle.Token => String = {
// case BlogTitle.NewLine => "" ← error: BlogTitle.NewLine doesn't exist
// case BlogComment.NewLine => "" ← error: BlogComment tokens not allowed
case BlogTitle.PlainText(txt) => txt
case BlogTitle.Link(url) => url
// Compiler warns missing BlogTitle.Quote(_) ✓
}
// General
val anyTokenToText: Base#Token => String = {
case a: PlainTextT#PlainText => a.text
case a: LinkT #Link => a.url
case a: NewLineT #NewLine => "\n"
// Compiler warns missing QuoteT#Quote ✓
}
// Recursive types
val t: BlogTitle .Quote => List[BlogTitle .Token] = _.content
val c: BlogComment.Quote => List[BlogComment.Token] = _.content
val g: QuoteT #Quote => List[Base #Token] = _.content
</pre>
<p>Look at that! That is <em>awesome</em>.
These are some things that we get:
</p>
<ul>
<li>No duplicated definitions or logic.</li>
<li>Generic & specific hierarchies are sealed, meaning the compiler will let
you know when you forget to cater for a case, or try to cater for a
case not allowed.</li>
<li>Children
of recursive types have the same specialisation.
<br/>Eg. a
<code>BlogTitle</code> can only quote using <code>BlogTitle</code> tokens.</li>
<li>Tokens can be processed generically.</li>
<li>Zero-overhead.
No additional computation or new memory allocation needed to store
tokens, or move them into a generic context. No
implicits.</li>
<li>Nice, neat pattern-matching which makes me happy.</li>
<li>It's just
plain ol' Scala traits so you're free to encode more constraints &
organisation. You can consolidate traits, add type aliases,
all that jazz.</li>
</ul>
<p>
There you go. Seems
like a great solution to this particular scenario.
</p>
<p>
Nothing is without downsides though. Creation will likely be a little hairy;
imagine writing a serialisation codec – the <code style="background: #eee;">Generic
⇒ Binary</code> part will be easy but <code style="background: #eee;">Binary
⇒ Specific</code> will be more effort. In my case I will only
create this data thrice {serialisation, parsing, random data
generation} but read and process it many, many times. Good tradeoff.
</p>
David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com9tag:blogger.com,1999:blog-5837865295676240265.post-25906613898029650502014-09-28T21:56:00.000+10:002014-09-28T21:56:08.610+10:00An Example of Functional Programming<div dir="ltr" style="text-align: left;" trbidi="on">
<div>
Many people, after
reading my <a href="http://japgolly.blogspot.com/2014/06/a-year-of-functional-programming.html">previous blog post</a>, asked to see a practical example of FP
with code. I know it's been a few months – I actually got married recently;
wedding planning is <i>very</i> time consuming – but I've finally come up with an
example. Please enjoy.</div>
<div>
<br /></div>
<div>
Most introductions
to FP begin with pleas for immutability but I'm going to do something
different. I've come up with a real-world example that's not too
contrived. It's about validating user data. There will be
5 incremental requirements that you can imagine coming in
sequentially, each one building on the other. We'll
code to satisfy each requirement incrementally without peeking at the
following requirements. We'll code with an FP mindset and use Scala
to do so but the language isn't important. This isn't about Scala.
It's about principals and a perspective of thought. You can write FP
in Java (if you enjoy pain) and you can write OO in Haskell (I know
someone who does this and baffles his friends). The language you use
affects the ease of writing FP, but FP isn't bound to or defined by
any one language. It's more than that. If you don't use Scala this
will still be applicable and useful to you.</div>
<div>
<br /></div>
<div>
I know many readers
will have programming experience but little FP experience so I
will try to make this as beginner-friendly as possible and omit using
jargon without explanation.</div>
<h2 style="margin-top: 2em;">
Req 1. Reject invalid input.</h2>
<div>
The premise here is
that we have data and we want to know if it's valid or not. For
example, suppose we want ensure a username conforms to certain rules
before we accept it and store it in our app's database.</div>
<div>
<br /></div>
<div>
I'm championing
<i>functional</i> programming here so let's use a function! What's the
simplest thing we need to make this work? A function that
takes some data and returns whether it's valid or not: <code style="background: #eee;">A ⇒ Boolean</code>.</div>
<div>
<br /></div>
<div>
Well that's
certainly simple but I'm going to handwaveily tell you that
primitives are dangerous. They denote the format of the underlying
data but not its meaning. If you refactor a function like
<code style="background: #eee;">blah(dataWasValid: Boolean, hacksEnabled: Boolean,
killTheHostages: Boolean)</code> the compiler isn't going to help you if you
get the arguments wrong somewhere. Have you ever had a bug where you
used the ID of one data object in place of another because they were
both <code>long</code>s? Did you hear about the NASA mission that failed because of
mixed metric numbers (eg. miles and kilometers) being
indistinguishable?</div>
<div>
<br /></div>
<div>
So let's address
that by first correcting the definition of our function. We want a
function that takes some data and returns <strike style="color: #aaaaaa;">whether it's valid
or not</strike> an indication of validity: <code style="background: #eee;">A ⇒ Validity</code>.</div>
<pre class="brush: scala">sealed trait Validity
case object Valid extends Validity
case object Invalid extends Validity
type Validator[A] = A => Validity</pre>
<div>
<br /></div>
<div>
We'll also create a
sample username validator and put it to use. First the validator:</div>
<pre class="brush: scala">val usernameV: Validator[String] = {
val p = "^[a-z]+$".r.pattern
s => if (p.matcher(s).matches) Valid else Invalid
}</pre>
<div>
<br /></div>
<div>
Now a sample save
function:</div>
<pre class="brush: scala">def example(u: String): Unit =
usernameV(u) match {
case Valid => println("Fake-saving username.")
case Invalid => println("Invalid username.")
}</pre>
<div>
<br /></div>
<div>
There's a problem
here. Code like this will make FP practitioners cry and for good
reason. How would we test this function? How could we ever manipulate
or depend on what it does, or its outcome? The problem here is
“<a href="http://en.wikipedia.org/wiki/Side_effect_(computer_science)" target="_blank">effects</a>” and unbridled, they are anathema to healthy, reusable
code. An effect is anything that affects anything outside the
function it lives in, relies on anything impure outside the function
it lives in, or happens in place of the function returning a value.
Examples are printing to the screen, throwing an exception, reading a
file, reading a global variable.</div>
<br/><div>
Instead we will
model effects as data. Where as the above example would either 1)
print <em>“Fake-saving username”</em> or 2) print <em>“Invalid username”</em>,
we will now either 1) return an effect that when invoked, prints
<em>“Fake-saving username”</em>, or 2) return a reason for failure.</div>
<div>
<br /></div>
<div>
We'll define our own datatype called <code>Effect</code>, to be a function that neither takes input nor
output.</div>
<div>
<em>(Note: If you're using Scalaz, <code style="background: #eee;">scalaz.effect.IO</code> is a decent catch-all for effects.)</em></div>
<pre class="brush: scala">type Effect = () => Unit
def fakeSave: Effect = () => println("Fake save")
</pre>
<div>
<br /></div>
<div>
Next, Scala provides
a type <code style="background: #eee;">Either[A,B]</code> which can be inhabited by either <code style="background: #eee;">Left[A]</code> or
<code style="background: #eee;">Right[B]</code> and we'll use this to return either an effect or failure
reason.</div>
<div>
Putting it all
together we have this:</div>
<pre class="brush: scala">def example(u: String): Either[String, Effect] =
usernameV(u) match {
case Valid => Right(fakeSave)
case Invalid => Left("Invalid username.")
}</pre>
<div>
<h2 style="margin-top: 2em;">
Req 2. Explain why input is invalid.</h2>
<div>
We need to specify a reason for failure now.</div>
<div>
We still have two
cases: valid with no error msg, invalid with an error msg. We'll
simply add an error message to the <code>Invalid</code> case.</div>
<pre class="brush: scala">case class Invalid(e: String) extends Validity</pre>
<div>
<br /></div>
<div>
Then we make it
compile and return the invalidity result in our example.</div>
<pre class="brush: diff"> val usernameV: Validator[String] = {
val p = "^[a-z]+$".r.pattern
- s => if (p.matcher(s).matches) Valid else Invalid
+ s => if (p.matcher(s).matches) Valid else
+ Invalid("Username must be 1 or more lowercase letters.")
}
def example(u: String): Either[String, Effect] =
usernameV(u) match {
case Valid => Right(fakeSave)
- case Invalid => Left("Invalid username.")
+ case Invalid(e) => Left(e)
}</pre>
<h2 style="margin-top: 2em;">
Req 3. Share reusable rules between validators.</h2>
<div>
Imagine
our system has 50 data validation rules, 80% reject empty strings,
30% reject whitespace characters, 90% have maximum string lengths. We
like reuse and D.R.Y. and all that; this requirement addresses that
by demanding that we break rules into smaller constituents
and reuse them.</div>
<div>
<br /></div>
<div>
We want to write small, independent units and join then into larger things.
This leads us to an important and interesting topic: composability.
</div>
<div>
<br /></div>
<div>
I want to suggest
something that I know will cause many people to cringe – but hear
me out – let's look to math. Remember basic arithmetic from ye olde
youth?</div>
<div style="margin-left: 4ex;">
8 = 5 + 3<br />
8 = 5 + 1 + 2</div>
<div>
Addition. This is great! It's building something from smaller parts. This
seems like a perfect starting point for composition to me.
There's a certain
beauty and elegance to math, and its capability is proven; what
better inspiration!</div>
<br />
<div>
Let's look at some basic properties of addition.</div>
<br />
<div style="font-weight: bold;">
Property #1:</div>
<div style="margin-bottom: 1em; margin-left: 4ex;">
<div>
8 = 8 + 0</div>
<div>
8 = 0 + 8</div>
<div>
Add 0 to any number and you get that number back unchanged.</div>
</div>
<div style="font-weight: bold;">
Property #2:</div>
<div style="margin-bottom: 1em; margin-left: 4ex;">
<div>
8 = (1 + 3) + 4</div>
<div>
8 = 1 + (3 + 4)</div>
<div>
8 = 1 + 3 + 4</div>
<div>
Parentheses don't matter. Add or remove them without changing the result.</div>
</div>
<div style="font-weight: bold;">
Property #3:</div>
<div style="margin-bottom: 1em; margin-left: 4ex;">
<div>
I'll also mention that in primary school, you had full confidence in this:</div>
<div style="margin-left: 2ex;">
<code>number + number = number</code></div>
<div>
It may seem silly to mention, but imagine if your primary school teacher told you that</div>
<div style="margin-left: 2ex;">
<code>number + number = number | null | InvalidArgumentException</code></div>
</div>
<div>
+ has other
properties too, like 2+6=6+2 but we don't want that for our
scenario with validation. The above three provide enough benefit for what
we need.</div>
<div>
<br /></div>
<div>
You might wonder why I'm describing these properties. Why should you care? Well as programmers you gain much by writing code with similar properties. Consider...</div>
<ul>
<li><div>
You know you don't
have to remember to check for nulls, catch any exceptions, worry
about our internal AddService™ being online.</div>
</li>
<li><div>
As long as the
overall order of elements is preserved, you needn't care about the order
in which groups are composed. i.e. we know that a+b+c+d+e will
safely yield the same result if we batch up execution of (a+b) and
(c+d+e) then add their results last. And parenthesis support is already
provided by the programming language.</div>
</li>
<li><div>
If ever forced
into composition by some code path and you can opt out by specifying
the 0 because we know that 0+x and x+0 are the same as x. No need to
overload methods or whatnot.</div>
</li>
</ul>
<div>
Simple right? Well have
you ever heard the term <i>“monoid”</i> thrown around? (Not “monad”.)
Guess what? We've just discussed all that makes a monoid what it is
and you learned it as a young child.</div>
<div>
A monoid is a binary
operation (x+x=x) that has 3 properties:</div>
<ul>
<li><b>Identity:</b> The 0 is what we call an identity element. 0+x = x = x+0</li>
<li><b>Associativity:</b> That's the ability to add/remove parentheses without changing the result.</li>
<li><b>Closure:</b> Always returns a result of the same type, no RuntimeExceptions, no nulls.</li>
</ul>
<div>
If jargon from
abstract algebra intimidates you, know that it's mostly just
terminology. You already know the concepts and have for years.
The knowledge is
very accessible and it's incredibly useful to be able to identify
these kinds of properties about your code.</div>
<div>
<br /></div>
<div>
Speaking of code,
let's implement this new requirement as a monoid.
We'll add
<code style="background: #eee;">Validator.+</code> for composition and ensure it preserves the associativity
property, and <code style="background: #eee;">Validator.id</code> for identity (also called zero).</div>
<div style="margin-top: 0.4em;">
<em>(Note: If using
Scalaz, Algebird or similar, you can explicitly declare your code to
be a monoid to get a bunch of useful monoid-related features for
free.)</em></div>
<pre class="brush: scala">case class Validator[A](f: A => Validity) {
@inline final def apply(a: A) = f(a)
def +(v: Validator[A]) = Validator[A](a =>
apply(a) match {
case Valid => v(a)
case e@ Invalid(_) => e
})
}
object Validator {
def id[A] = Validator[A](_ => Valid)
}</pre>
<div>
The difficulty of building
human-language sentences scales with expressiveness. For our demo
it's enough to simply have validators contain error message clauses
like <em>“is empty”</em>, <em>“must be lowercase”</em> and just tack the
subject on later.</div>
<div>
<br /></div>
<div>
First we define some
helper methods <code style="background: #eee;">pred</code> and <code style="background: #eee;">regex</code>, then use them to create our validators</div>
<pre class="brush: scala">object Validator {
def pred[A](f: A => Boolean, err: => String) =
Validator[A](a => if (f(a)) Valid else Invalid(err))
def regex(r: java.util.regex.Pattern, err: => String) =
pred[String](a => r.matcher(a).matches, err)
}
val nonEmpty = Validator.pred[String](_.nonEmpty, "must be empty")
val lowercase = Validator.regex("^[a-z]*$".r.pattern, "must be lowercase")
val usernameV = nonEmpty + lowercase</pre>
<div>
Then we gaffe our subject to on to our error messages before displaying it and we're done.</div>
<pre class="brush: scala">def buildErrorMessage(field: String, err: String) = s"$field $err"
def example(u: String): Either[String, Effect] =
usernameV(u) match {
case Valid => Right(fakeSave)
case Invalid(e) => Left(buildErrorMessage("Username", e))
}</pre>
<h2 style="margin-top: 2em;">
Req 4. Explain all the reasons for rejection.</h2>
<div>
Users
are complaining that they get an error message, fix their data
accordingly only to have it then rejected for a different reason.
They again fix their data and it is rejected again for yet another
reason. It would be better to inform the user of all the things left
to fix so they can amend their data in one shot.</div>
<div>
For
example, an error message could look like <em>“Username 1) must be less than 20 chars, 2) must contain at least one number.”</em></div>
<div>
<br /></div>
<div>
In other words there
can be 1 or more reasons for invalidity now. Ok, we'll amend <code>Invalid</code>
appropriately...</div>
<pre class="brush: scala">case class Invalid(e1: String, en: List[String]) extends Validity</pre>
<div>
<br /></div>
<div>
Then we just make the compiler happy...</div>
<pre class="brush: diff"> case class Validator[A](f: A => Validity) {
@inline final def apply(a: A) = f(a)
def +(v: Validator[A]) = Validator[A](a =>
- apply(a) match {
- case Valid => v(a)
- case e@ Invalid(_) => e
- })
+ (apply(a), v(a)) match {
+ case (Valid , Valid ) => Valid
+ case (Valid , e@ Invalid(_,_)) => e
+ case (e@ Invalid(_,_), Valid ) => e
+ case (Invalid(e1,en) , Invalid(e2,em) ) => Invalid(e1, en ::: e2 :: em)
+ })
}
object Validator {
def pred[A](f: A => Boolean, err: => String) =
- Validator[A](a => if (f(a)) Valid else Invalid(err))
+ Validator[A](a => if (f(a)) Valid else Invalid(err, Nil))
}
-def buildErrorMessage(field: String, err: String) = s"$field $err"
+def buildErrorMessage(field: String, h: String, t: List[String]): String = t match {
+ case Nil => s"$field $h"
+ case _ => (h :: t).zipWithIndex.map{case (e,i) => s"${i+1}) $e"}.mkString(s"$field ", ", ", ".")
+}
def example(u: String): Either[String, Effect] =
usernameV(u) match {
case Valid => Right(fakeSave)
- case Invalid(e) => Left(buildErrorMessage("Username", e))
+ case Invalid(h, t) => Left(buildErrorMessage("Username", h, t))
}
</pre>
<div>
<em>(Note:
If you're using Scalaz, <code style="background: #eee;">NonEmptyList[A]</code> is a better replacement for
<code style="background: #eee;">A, List[A]</code> like I've done in <code style="background: #eee;">Invalid</code>. The same thing can also be
achieved by <code style="background: #eee;">OneAnd[List, A]</code>. In fact <code style="background: #eee;">OneAnd</code> is a good way to have
compiler-enforced non-emptiness.)</em></div>
<h2 style="margin-top: 2em;">
Req 5. Omit mutually-exclusive or redundant error messages.</h2>
<div>
Take
the this error message: <em>“Your name must 1) include a given name, 2)
include a surname, 3) not be empty”</em>. If the user forgot to enter
their name you just want to say <em>“hey you forgot to enter your
name”</em>, not bombard the user with details about potentially invalid
names.</div>
<div>
<br /></div>
<div>
What does this mean?
It means one rule is unnecessary if another rule fails. What we're
really talking about here is the means by which rules are composed.
Let's just add another composition method. We talked about the +
operation in math already, well math also provides a multiplication
operation too. Look at an expression like 6 + 14 + (7 * 8). Two types
of composition, us explicitly clarifying our intent via parentheses.
That's perfectly expressive to me and it solves our new requirement
with simplicity and minimal dev. As a reminder that we can borrow
from math without emulating it verbatim, instead of a symbol let's
give this operation a wordy name like <code style="background: #eee;">andIfSuccessful</code> so that
we can say <code style="background: #eee;">nonEmpty andIfSuccessful containsNumber</code> to indicate
a validator that will only check for numbers if data isn't empty.</div>
<div>
<br /></div>
<div>
Just like these
express different intents and yield different results</div>
<div style="margin-left: 4ex;">
<div>
number = 4 * (2 + 10)</div>
<div>
number = (4 * 2) + 10</div>
</div>
<div>
So too can</div>
<div style="margin-left: 4ex;">
<div>
<span style="color: purple;">rule</span> = <span style="color: purple;"><span lang="zxx">nonEmpty</span></span><span lang="zxx">
</span><span lang="zxx">andIfSuccessful </span><span lang="zxx"><b>(</b></span><span style="color: purple;"><span lang="zxx">containsNumber</span></span><span lang="zxx">
and </span><span style="color: purple;"><span lang="zxx">isUnique</span></span><span lang="zxx"><b>)</b></span></div>
<div>
<span style="color: purple;">rule</span> = <b>(</b><span style="color: purple;">nonEmpty</span>
andIfSuccessful <span style="color: purple;">containsNumber</span><b>)</b>
and <span style="color: purple;">isUnique</span></div>
</div>
<div>
Or if you don't mind custom operators</div>
<div style="margin-left: 4ex;">
<div>
<span style="color: purple;">rule</span> = <span style="color: purple;">nonEmpty</span>
<b>>></b> <b>(</b><span style="color: purple;">containsNumber</span> <b>+</b>
<span style="color: purple;">isUnique</span><b>)</b></div>
<div lang="zxx" style="font-weight: normal; line-height: 100%; margin-bottom: 0cm;">
<span style="color: purple;">rule</span> = <b>(</b><span style="color: purple;">nonEmpty</span>
<b>>></b> <span style="color: purple;">containsNumber</span><b>)</b> <b>+</b>
<span style="color: purple;">isUnique</span></div>
</div>
<br />
<div>
To implement this new requirement we add a single method to <code>Validator</code>:</div>
<pre class="brush: scala">def andIfSuccessful(v: Validator[A]) = Validator[A](a =>
apply(a) match {
case Valid => v(a)
case e@ Invalid(_,_) => e
})</pre>
<h2 style="margin-top: 2em;">
Conclusion</h2>
<div>
And we're done.<br />
It's not how I would've approached code years back in my OO/Java era, nor is it like any of the code I came across written by others in that job. As an experiment I started fulfilling these requirements in Java the way old me used to code and there was a <i>loooot</i> of wasted code between requirements. I'd get all annoyed at each new step, so much so that I didn't even bother finishing. On the contrary, I <i>enjoyed</i> writing the FP.<br />
<br />
Right, what conclusions can we draw?<br />
<br />
<b>FP is simple.</b> Each validation is a single function in a wrapper.</div>
<div>
<b>FP is flexible.</b> Logic is reusable and can be assembled into complex
expressions easily.</div>
<div>
<b>FP is easily maintainable & modifiable.</b> It has less structure, less
structural dependencies, and is less code, plus the compiler's got your back.</div>
<div>
<b>FP is easy on the author.</b> There was next to no rewriting or throw-away of
code between requirements, and each new requirement was easy to
implement.</div>
<div>
<br /></div>
<div>
I hope this proves an effective concrete example of FP for programmers of different backgrounds. I also hope this
enables <i>you</i> to write more reliable software and have a happier
time doing it.<br />
<br /></div>
<div>
Go forth and
function.<br />
<br />
<div style="text-align: left;">
<i><a href="https://gist.github.com/japgolly/fe822c84855da9b7f011" target="_blank">(Source code)</a></i></div>
</div>
</div>
</div>David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com7tag:blogger.com,1999:blog-5837865295676240265.post-59069409801605832152014-06-09T18:53:00.000+10:002014-06-09T20:50:15.653+10:00A Year of Functional Programming<div dir="ltr" style="text-align: left;" trbidi="on">
<div>
It's been a year
since I first came across the concept of functional programming. To
say it's changed my life, is an unjust understatement. This is a
reflection on that journey.</div>
<div>
<br /></div>
<div>
<i>Warning: I use the term FP quite loosely throughout this article.</i></div>
<div>
<br /></div>
<h3>Where I Was</h3>
<div style="font-weight: normal; margin-bottom: 0cm;">
I've been coding since the age of 8, so 26 years now. I started with
BASIC & different types of assembly then moved on to C, C++, PHP
and Perl <i>(those were different times, maaaan ☮)</i>, Java, JavaScript,
Ruby. That's the brief gist of it. <span style="line-height: 120%;">Basically: a lot of time on the clock and absolutely no exposure to
FP concepts or languages. I thought functional programming just meant
recursive functions (<i>ooooo big deal right?</i>). I really came in blind.</span></div>
<div style="font-weight: normal; margin-bottom: 0cm;">
<br /></div>
<h3>How It Started: Scala</h3>
<div>
Last year, I wanted
the speed and static checking (<i>note: not “types”</i>) of Java with
the conciseness and flexibility of Ruby. I came across Scala, skimmed
a little and was impressed. I bought a copy of <u><a href="http://www.amazon.com/Scala-Impatient-Cay-S-Horstmann-ebook/dp/B007JWDMIE" target="_blank">Scala For The Impatient</a></u> and just ate it for breakfast. I read the entire thing
in 2 or 3 days, jotted down everything useful and then just started
coding. It was awesome! At first I was just coding the same way I
would Java with less than half the lines of code. It is a very
efficient Java.</div>
<div>
<br /></div>
<h3>Exposure: Haskell</h3>
<div style="font-weight: normal; margin-bottom: 0cm;">
Lurking around the Scala community, I came across a joke. Someone
said “Scala is a gateway drug to Haskell.” I found that amusing,
although not for the reasons that the author intended. Haskell? Isn't
that some toy university language? An experiment or something. Is it
even still alive? Scala's awesome and so powerful, why would <i>that</i>
lead to Haskell? How.... intriguing. Inexplicably it piqued my
interest and really stuck with me. Later I decided to look it up and
yes, it sure was alive and very active. I was shocked to discover
that it compiles to machine code (binary) and is comparable in speed to C/C++.
<i>What?!</i> It seems idiotic now but a year ago I thought it was some
interpreted equation solver. I'm not alone in that ignorance, sadly;
talking about it to some mates over lunch last year and a friend
incredulously burst out laughing, “Haskell?” as if I was trying
to tell him my dishwasher had an impressive static typing system. It
saddens me to realise how in the dark I was, and how many people
still are. Haskell is pretty frikken awesome! True to the gateway drug prophecy, I do now look to
Haskell as an improvement to using Scala. But let's get back on
track.</div>
<div style="font-weight: normal; margin-bottom: 0cm;">
<br /></div>
<h3>Exposure: Scalaz</h3>
<div style="font-weight: normal; margin-bottom: 0cm;">
I also started seeing contentious discussion of some library called
<a href="https://github.com/scalaz/scalaz" target="_blank">Scalaz</a>. Curious, I had a look at the code to see what people were on
about, and didn't understand it at all. I'd see classes like
<code>Blah[F[_], A, B]</code>, methods with confusing names that take params like
<code>G[_], A → C, F[X → G[Y]]</code>, implementations like
<code>F.boo_(f(x))(g)(x)</code>, and I'd just think “<i>What the hell is this? How
is this useful?</i>”. I was used to <i>methods</i> that did something
pertaining to a goal in its domain. This Scalaz code was very alien
to me and yet, very intriguing. Some obviously-smart person spent
time making the alphabet soup permutations, why?</div>
<div style="font-weight: normal; margin-bottom: 0cm;">
<br /></div>
<div style="font-weight: normal; margin-bottom: 0cm;">
I've since discovered that answer to that question and I never
could've imagined the amount of benefit it would yield. Instead of
methods with domain-specific meaning, I now see functions for
morphisms, in accordance with relevant laws. Simply put: changing the
shape or content of types. No mention of any kind of
problem-domain; applicable to all problem-domains! It's been
surprising over the last year to discover just how applicable this
is. This kind of abstraction is prolific, it's in your code right
now, disguised, and intertwined with your business logic. Without
knowledge of these kind of abstractions (and the awareness that a
type-component level of abstraction <i>is</i><span style="font-style: normal;">
indeed possible) you are doomed to reinvent the wheel for all time
and not even realise it. </span><span style="font-style: normal;">Identifying
and using these abstractions, your </span>code becomes more reusable,
simple, readable, flexible, and testable. When you learn to recognise
it, it's breathtaking.</div>
<div>
<br /></div>
<h3>FP: The Basics</h3>
<div style="font-weight: normal; margin-bottom: 0cm;">
Now that I'd been exposed to FP I started actively learning about it.
At first I learned about referential transparency, purity and
side-effects. I nodded agreement but had major reservations about the
feasibility of adhering to such principals, and so at first I wasn't
really sold on it. Or rather, I was hesitant. I may have been guilty
of mumbling sentences involving the term “real-world”. Next came
immutability. Now I'm a guy who used to use final religiously in
Java, const on everything in C++ back in the day, and here FP is
advocating for data immutability. Not just religiously advocating but
providing real, elegant solutions to issues that you encounter using
fully immutable data structures. Wow! So with immutability,
composability, lenses it had its hooks in me.</div>
<div style="font-weight: normal; margin-bottom: 0cm;">
<br /></div>
<div style="font-weight: normal; margin-bottom: 0cm;">
Next came advocation for more expressive typing and (G)ADTs. That
appealed in theory too and again I was hesitant about its
feasibility. Once I experimentally applied it to some parts of my
project, I was blown away by how well it worked. That became the
gateway into thinking of code/types algebraically, which lead to...</div>
<div style="font-weight: normal; margin-bottom: 0cm;">
<br /></div>
<h3>FP: Maths</h3>
<div style="font-weight: normal; margin-bottom: 0cm;">
I loved maths back in school and always found it easy. Reading FP
literature I started coming across lots of math and at first thought
“<i>great! I'm awesome at maths!</i>” but then, trying to make sense of
some FP math stuff, I'd find myself spending hours clicking link
after link, realising that I wasn't getting it and, in many cases,
still couldn't even make sense of the notation. It became daunting.
Even depressing. Frequently demotivating.</div>
<div style="font-weight: normal; margin-bottom: 0cm;">
<br /></div>
<div style="font-weight: normal; margin-bottom: 0cm;">
The good news is that everything you need is out there; you just have
to be prepared to learn more than you think you need. I persisted, I
stopped viewing it as an annoying bridge and starting treating it as
a fun subject on its own and, before long things made sense again. It
<span style="font-style: normal;">opens new doors when you </span><span style="font-style: normal;">learn
it</span><span style="font-style: normal;">.</span></div>
<div style="font-weight: normal; margin-bottom: 0cm;">
<br /></div>
<div>
Example: I had a
browser tab (about co-Yoneda lemma) open for 3 months because I
couldn't make sense of it. It took months (granted not everyday) of
trying then confusion then tangents to understand background and
whatever it was that threw me off. Once I learned that final piece of
background info, I went from understanding only the first 5% to 100%.
It was a great feeling.</div>
<div style="font-style: normal; font-weight: normal; margin-bottom: 0cm;">
<br /></div>
<div style="font-style: normal; font-weight: normal; margin-bottom: 0cm;">
<br /></div>
<h3>Feeling Intimidated</h3>
<div style="font-weight: normal; margin-bottom: 0cm;">
Looking back there were times when I felt learning FP quite
intimidating. When I'm in/around/reading conversations between
experienced FP'ers quite often I've seriously felt like a moron. I
started wondering if I gave my head a good slap, would a potato fall
out. It can be intimidating when you're not used to it. But really,
my advice to you, Reader, is that everyone's nice and happy to help
when you're confused. I have a problem asking for help but I've seen
everyone else do it and be received kindly... then I swoop in an
absorb all the knowledge, hehe.</div>
<div style="font-weight: normal; margin-bottom: 0cm;">
<br /></div>
<div style="font-weight: normal; margin-bottom: 0cm;">
It's a mindset change. I wish I'd known this earlier as it would've
saved me frustration and doubt, but you kind of need to unlearn what
you think you know about coding, then go back to the basics. Imagine
you've driven trains for decades, and spontaneously decide you want
to be a pilot. No, you can't just read a plane manual in the morning
and be in Tokyo in the afternoon. No, if you grab a beer with
experienced pilots you won't be able to talk about aviation at their
level. It's normal, right? Be patient, learn the basics, have fun,
you'll get there.</div>
<div style="font-weight: normal; margin-bottom: 0cm;">
<br /></div>
<div style="font-weight: normal; margin-bottom: 0cm;">
On that note, I highly recommend <a href="http://www.manning.com/bjarnason/" target="_blank">Fuctional Programming in Scala</a>, it's a phenomenal book. It helped me wade my
way from total confusion to comfortable comprehension on a large
number of FP topics with which I was struggling trying to learn from
blogs.</div>
<div style="font-weight: normal; margin-bottom: 0cm;">
<br /></div>
<div>
<br /></div>
<h3>Realisation: Abstractions</h3>
<div style="font-weight: normal; margin-bottom: 0cm;">
Recently I looked at some code I wrote 8 months ago and was shocked!
I looked at one file written in “good OO-style”, lots of
inheritance and code reuse, and just thought “<i>this is just a monoid
and a bunch of crap because I didn't realise this is a monoid</i>” so I
rewrote the entire thing to about a third of the code size and ended up
with double the flexibility. Shortly after I saw another file and
this time thought “<i>these are all just endofunctors,</i>” and lo and
behold, rewrote it to about a third of the code size and the final
product being both easier to use and more powerful.</div>
<div style="font-weight: normal; margin-bottom: 0cm;">
<br /></div>
<div style="font-weight: normal; margin-bottom: 0cm;">
I now see more abstractions than I used to. Amazingly, I'm also
starting to see similar abstractions outside of code, like in UI
design, in (software) requirements. It's brilliant! If you're not
on-board but aspire to write “DRY” code, you will love this.</div>
<div style="font-weight: normal; margin-bottom: 0cm;">
<br /></div>
<h3>Realisation: Confidence. Types vs Tests</h3>
<div>
I require a degree
of confidence in my code/system, that varies with the project. I do
whatever must be done to achieve that. In Ruby, that often meant
testing it from every angle imaginable, which cost me significant
effort and negated the benefit of the language itself being concise.
In Java too, I felt the need to test rigorously.</div>
<div>
<br /></div>
<div>
At first I was the
same in Scala, but since learning more FP, I test way less and have
more confidence. Why? The static type system. By combining
lower-level abstractions, an expressive type system, efficient and
immutable data structures, and the laws of parametricity, in most
cases when something compiles, it works. Simple as that. There are
hard proofs available to you, I'm not talking about fuzzy feelings
here. I didn't have much respect for static types coming from Java
because it's hard to get much mileage out of it <i><span style="color: #999999;">(even in Java 8 –
switch over enum needs a default block? Argh fuck off! Gee
maybe maybe all interfaces should have a catch-all method too then.
That really boiled my blood the other day. Sorry-)</span></i>, anyway: Java as a
static typing system is like an abusive alcoholic as a parent. They
may put food on the table and clothes your back, but that's a far cry
from a good parent. (And you'll become damaged.) Scala on the other
hand teaches you to trust again. Trust. Trust in the compiler. I've
come to learn that when you trust the compiler and can express
yourself appropriately, entire classes of problems go away, and with
it the need for testing. It's joyous.</div>
<div>
<br /></div>
<div>
Sadly though,
eventually you get to a point where Scala starts to struggle. It gets
stupid, it can't work out what you mean, what you're saying, you have
to gently hold its hand and coax it with explicit type declarations
or tricks with type variance or rank-n types. Once you get to that
level you start to feel like you've outgrown Scala and now need a big
boy's compiler which can lead to habitual grumbling and regular
reading about Haskell, Idris, Agda, Coq, et al.</div>
<div style="font-weight: normal; margin-bottom: 0cm;">
<br /></div>
<div style="font-weight: normal; margin-bottom: 0cm;">
However when you do need tests, you can write a single test for a
bunch of functions using a single expression. How? Laws. Properties.
Don't know what I mean? Pushing an item on to a stack should always
increase its size by 1, the popping of which should reduce its size
by 1 and return the item pushed, and return a stack equivalent to
what you started with. Using libraries like <a href="http://www.scalacheck.org/">ScalaCheck</a>, turning that
into a single expression like <code>pop(push(start, item)) == (start, item)</code>
which is essentially all you need to write; ScalaCheck will generates
test data for you.</div>
<div style="font-weight: normal; margin-bottom: 0cm;">
<br /></div>
<div>
<br /></div>
<h3>Where Next?</h3>
<div>
What does the future
hold for me? Well, I could never go back to dynamically-typed
language again.</div>
<div>
<span style="font-style: normal;">I
</span><span style="font-style: normal;">will
stick with Scala as I've invested a lot in it and it's still the best
language I know well. I'd like to get more hands-on experience with
<a href="http://learnyouahaskell.com/">Haskell</a>; I don't know its trade-offs that well but its type system
</span><span style="font-style: normal;">seems angelic. </span><span style="font-style: normal;">Go</span><span style="font-style: normal;">t
my eye on <a href="http://www.idris-lang.org/">Idris</a>, </span><span style="font-style: normal;">too</span><span style="font-style: normal;">.</span></div>
<div>
<br /></div>
<h3>Academia!</h3>
<div>
I used to get
excited discovering new libraries. I'd always think “Great! I
wonder what this will allow me to do.” Well now I feel that way
about research & academic papers. They are the same thing except
smarter, more lasting, more dependable, and they yield more
flexibility. It's awesome and I've got decades of catching up to do!
Over the next year I'll definitely spend a lot of time learning more FP and comp-sci theory.
I'd also like to be able to understand most Haskell blogs I come across. They promise very intelligent constructs (which aren't specific to Haskell) but the typing usually gets a bit too dense; it'd be nice to be able to read those articles with ease.
</div>
<div>
<br /></div>
<div>
<u>Don't fall for
the industry dogma</u> that academia isn't applicable to the
real-world. What a load of horseshit that lie is. It <i>does</i><span style="font-style: normal;">
apply to the real-world, it</span><span style="font-style: normal;">'s
here to </span><i>help </i><span style="font-style: normal;">you
achieve your goals, </span><span style="font-style: normal;">it will
</span><i>save</i><span style="font-style: normal;"> you significant
time and effort, even with the initial learning overhead considered</span><span style="font-style: normal;">.
</span><span style="font-style: normal;">Don't say you don't have time
to do things in half the time. </span><span style="font-style: normal;">If
you're always busy and your business are super fast-paced agile scrum
need-it-yesterday kinda people, well I know you don't have time, but
what I'm offering is this: say you just need to get something out the
door and can do it quickly and messily </span><span style="font-style: normal;">in
1000 lines in 6 hours with 10 bugs and </span><span style="font-style: normal;">10</span><span style="font-style: normal;">
“abilities”, </span><span style="font-style: normal;">well if you
spend a bit of your own time learning you could perform the same task
in </span><span style="font-style: normal;">400 lines in 4 hours with
maybe 1 bug and </span><span style="font-style: normal;">2</span><span style="font-style: normal;">0
“abilities”. </span><span style="font-style: normal;">You've just
saved 2 hours up-front, not to mention days of savings when adding
new features, fixing bugs, etc. That's applicable to you, the
“real-world” and the industry. </span><span style="font-style: normal;">I've
spent years in the industry and </span><span style="font-style: normal;">not
just as a coder </span><span style="font-style: normal;">and I wish
I'd known about this stuff back then because it would've saved me so
much time, effort and stress. </span><span style="font-style: normal;">There
seems to be this odd disdain for academia throughout the industry.
</span><span style="font-style: normal;">Reject it. It's an ignorant
justification of laziness and short-sightedness. It's false. I
encourage you to take the leap.</span></div>
<br />
<div>
<br /></div>
</div>David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com36tag:blogger.com,1999:blog-5837865295676240265.post-78362829515554645902013-10-26T11:06:00.000+11:002013-10-26T11:16:33.889+11:00Scala: Methods vs Functions<p>
It's a bright, windy Saturday morning. Sipping a nice, warm coffee I find myself musing over the performance implications of subtleties between methods and functions in Scala. Functions are first-class citizens in Scala and represented internally as instances of <code>Function<em>n</em></code> (where n is the arity); effectively, an interface with an <code>apply</code> method. Methods are methods are methods; they are directly invocable in JVM-land.
</p>
<p>
So what differences are there that could affect performance?
I can think of:
<ul>
<li>Because functions are traits with abstract type members, in JVM-land those abstract types will be erased to Objects. I presume this means boxing for your primitives like int and long (unless scalac has any tricks up its sleeve like <code>@specialized</code>)
</li>
<li>When passing a method to a higher-order function, methods will need to be boxed into a Function. For example, in <blockquote><code>def method(s:String) = s.length<br>List("abc").map(method)</code></blockquote>
the <code>map</code> method requires an instance of <code>Function1</code> so I (again, presume) the compiler generates a synthetic instance of <code>Function1</code> as a proxy to the target method.
</li>
<li>The JVM can invoke methods directly. To invoke a function, it will have to first load up the Function object and then invoke its <code>apply</code> method. That's an extra hop.</li>
<li>Probably more.</li>
</ul>
</p>
<p>
So those are some of the differences between functions and methods in Scala. Let's see how they perform. There's an awesome little micro-benchmarking tool called <a href="http://axel22.github.io/scalameter/">ScalaMeter</a>. It takes about 2 min to get started with it. I decided to test 1,000,000 reps along three axes:
<ol>
<li>Direct invocation vs passing to someone else</li>
<li>Primitive vs Object argument</li>
<li>Primitive vs Object result</li>
</ol>
</p>
<h1>The Results</h1>
<p>
<table class="withBorders" style="margin-left:0; margin-right:0">
<tr><th></th><th></th><th>Fn improvement over Method</th></tr>
<tr><th rowspan=4>Direct</th><th>int → int</th><td style="text-align:right; color:red">-6.32%</td></tr>
<tr><th>int → str</th><td style="text-align:right; color:red">-9.53%</td></tr>
<tr><th>str → int</th><td style="text-align:right; color:red">-4.39%</td></tr>
<tr><th>str → str</th><td style="text-align:right; color:#811">-1.09%</td></tr>
<tr><th rowspan=4>As Arg</th><th>int → int</th><td style="text-align:right">0.31%</td></tr>
<tr><th>int → str</th><td style="text-align:right">0.94%</td></tr>
<tr><th>str → int</th><td style="text-align:right">-0.22%</td></tr>
<tr><th>str → str</th><td style="text-align:right">-0.24%</td></tr>
</table>
What can we see?
<ul>
<li>It seems that there <u>is</u> a boxing cost for functions' i/o.</li>
<li>It seems that there <u>is</u> a <em>slight</em> cost when invoking functions directly.</li>
<li>It seems that there <u>is no</u> real cost boxing methods into functions.</li>
</ul>
<h1>Conclusion</h1>
<p style="margin-top:0">
If you're like one-week-ago-me <em>(pft he's idiot!)</em>, you might think you're helping the compiler out by writing functions instead of methods when their only use is to be passed around. Well it doesn't appear to be so.<br>
Just use methods and let your mind (if you're lucky enough to have its cooperation) worry about and solve other things.
</p>
<p style="margin-top:3em">
Code and raw results available here: <a href="https://github.com/japgolly/misc/tree/methods_and_functions">https://github.com/japgolly/misc/tree/methods_and_functions</a>
</p>David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com2tag:blogger.com,1999:blog-5837865295676240265.post-63016945591280756542013-09-05T14:56:00.000+10:002013-09-05T14:56:17.752+10:00Keyed Lenses<p><em>
<strong>TL;DR:</strong> Lenses are cool. I've come up with keyed lenses which I find helpful. Hopefully you will too. Do you? Have I just reinvented the wheel in ignorance?
</em></p>
<p>
Annual blog post time. So this year I've been imploding and exploding with enthusiasm over functional programming. Already I've found some FP perspectives and strategies mind-boggle-blow-blast-ingly effective, and beautiful. My day project is in a significantly better state owing to FP. If you're not onboard I recommend reading <a href="http://learnyouahaskell.com/chapters">Learn You a Haskell for Great Good!</a> and <a href="http://www.manning.com/bjarnason/">Functional Programming in Scala</a>.
</p>
<p>
<em>(Btw: big thanks to NICTA, specifically <a href="https://twitter.com/dibblego">Tony Morris</a> & <a href="https://twitter.com/markhibberd">Mark Hibberd</a> who held 2 free FP courses and gracefully tolerated numerous dumb-shit moments from me. I think it's a semi-annual recurring thing so keep an eye out on <a href="https://groups.google.com/forum/#!forum/scala-functional">scala-functional</a> for the next one if you're interested and in Australia.)</em>
</p>
<p>
<strong>What is a lens?</strong> If you don't know what a lens is, it's basically a decoupled getter/setter that be composed with other lenses, so that the depth and structure of data can be hidden. In traditional OO you might not see the merits but when your data structures are all immutable, the benefit is immense. There are plenty of good resources online to learn more, such as <a href="http://www.parleys.com/play/51c387cae4b0ed8770356869/">this</a>, <a href="http://www.monadzoo.com/blog/2012/11/18/using-lenses-with-scalaz-7/">this</a> and <a href="https://blog.stackmob.com/2012/02/an-introduction-to-lenses-in-scalaz/">this</a>.
</p>
<h1>KeyedLenses</h1>
<p>
What I'm calling a KeyedLens, is a lens that points to a value in a composite value such that a key is required. A Map is an obvious example.
</p>
<p>
<em>(NOTE: Scalaz has some basic support for this -- I am aware of it -- but I find that it doesn't my needs and/or the way I try to use it. I find that the call-site syntax becomes long and nasty, it doesn't compose well, and it creates new lenses every time it's used which is inefficient).</em>
</p>
<p>
Let's start with some toy data.<br/>
I'm going to use Scala and the awesome <a href="https://github.com/scalaz/scalaz">Scalaz</a> library, and there's a link to the KeyedLens source code at the end of the post. (If you don't know Scala, just imagine it's pseudo-code. The concepts translate into almost anything.)
</p>
<p>
Let's model a band from a guitarist's point of view.
<script src="https://gist.github.com/japgolly/6435963.js"></script>
</p>
<p>
Here we're modelling the <a href="http://www.youtube.com/watch?v=pPBTtsgIWNM">mighty band Tesseract</a> (think Pink Floyd + Meshuggah).<br/>
There are two places where I'm going to use a keyed lens.
<ol>
<li>To access the guitar of a given band member. <em>(guitarL)</em></li>
<li>To access the string gauge of a given guitar. <em>(stringGaugeL)</em></li>
</ol>
</p>
<p>
Here are the lens definitions:
<script src="https://gist.github.com/japgolly/6436340.js"></script>
</p>
<p>
This gives us the following lenses:
<table class="withBorders" style="margin-left:0; margin-right:0">
<tr><th>LENS</th><th>TYPE</th></tr>
<tr><td>guitarTuningL</td><td><code>LensFamily[Guitar, Guitar, String, String]</code></td></tr>
<tr><td>stringGaugeL</td><td><code>LensFamily[(Guitar,Int), Guitar, Double, Double]</code></td></tr>
<tr><td>bandNameL</td><td><code>LensFamily[Band, Band, String, String]</code></td></tr>
<tr><td>guitarL</td><td><code>LensFamily[(Band,Person), Band, Guitar, Guitar]</code></td></tr>
<tr><td style="border:solid 1px #222; padding:2px">guitaristsTuningL</td><td><code>LensFamily[(Band,Person), Band, String, String]</code></td></tr>
<tr><td>guitaristsGaugeL</td><td><code>LensFamily[(Band,(Person,Int)), Band, Double, Double]</code></td></tr>
</table>
Notice the keys always get propagated to the left.</br>
Now let's see them in action. This is what appeals to me the most.
</p>
<h1>Usage. The Fun Part.</h1>
<p>
<strong>Get with one key.</strong> <em>What is Acle's guitar tuned to?</em>
<pre style="background:#eee"><code>scala> guitaristsTuningL.get(band, acle)
res0: String = BEADGBE</code></pre>
</p>
<p>
<strong>Get with two keys.</strong> <em>What is the gauge of Acle's 7th string?</em>
<pre style="background:#eee"><code>scala> guitaristsGaugeL.get(band, (acle, 7))
res1: Double = 0.059</code></pre>
</p>
<p>
<strong>Set with one key.</strong> <em>I want to change Acle's tuning.</em>
<pre style="background:#eee"><code>scala> guitaristsTuningL.set((band, acle), "G#FA#D#FA#D#")
res2: Band = Band(Tesseract,Map(Person(Acle) -> Guitar(7,G#FA#D#FA#D#,List(0.011, 0.014, 0.018, 0.028, 0.038, 0.049, 0.059)), Person(James) -> Guitar(6,EADGBE,List(0.01, 0.013, 0.017, 0.026, 0.036, 0.046))),Set(Person(Jay), Person(Amos), Person(Ashe)))
</code></pre>
</p>
<p>
<strong>Set with two keys.</strong> <em>I want to lower the gauge of Acle's 7th string.</em>
<pre style="background:#eee"><code>scala> guitaristsGaugeL.set((band, (acle, 7)), 0.0666666)
res3: Band = Band(Tesseract,Map(Person(Acle) -> Guitar(7,BEADGBE,List(0.011, 0.014, 0.018, 0.028, 0.038, 0.049, 0.0666666)), Person(James) -> Guitar(6,EADGBE,List(0.01, 0.013, 0.017, 0.026, 0.036, 0.046))),Set(Person(Jay), Person(Amos), Person(Ashe)))
</code></pre>
</p>
<p style="text-align:right"><em>[If you're new to lenses, keep in mind that all the data here is immutable. Objects are copied and reused.]</em></p>
<p>
<ul>
<li>I really like the brevity of these lines.</li>
<li>I like that you get a single lens that <em>requires</em> a key be provided.</li>
<li>I don't like the <code>(A, (K1,K2))</code> type of <code>guitaristsGaugeL</code> which is easily changable into <code>(A, K1, K2)</code> but then what happens when it's further recomposed? I'd probably need methods like compose2, compose3, compose4, etc. Will think later.</li>
</ul>
What do you think? Does anything like this already exist?
</p>
<p style="font-size:120%">
<strong>Source code</strong> <a href="https://gist.github.com/japgolly/6436598">for KeyedLenses.scala is here</a>.
</p>
<p>Thanks!</p>David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com3tag:blogger.com,1999:blog-5837865295676240265.post-41915470863229954822013-04-25T11:57:00.003+10:002013-04-25T12:02:59.339+10:00Trial: Choosing Lift over Rails<p>
Today, I'm going to start on a slightly scary journey. I'm going to start work on a new webapp but I'm not going to use Ruby on Rails which I (mostly) love (with plugins) and already know well. Instead I'm going to use the Scala-based <a href="http://liftweb.net/">Lift framework</a> which I have never used and is completely alien to me. This is going to cost me a lot of time at the beginning, not just because I don't know the API, but because the mindset of the beast seems drastically different to typical web frameworks that most of us are used to.
</p>
<p><em>(<b>NOTE re RAILS:</b> I will make many comparisons to RoR because it's the web framework I know best and cos it's well-known. Just like everyone thinks Maccas (ie. McDonalds for non-Aussies) when you think fast-food. It's tough being on top. Bad luck RoR.)</em></p>
<p><em>(<b>Have you made a similar transition?</b> I'd love to know how it went. <a href="mailto:japgolly@gmail.com">Let me know</a>!</em>)</p>
<h1>Why?</h1>
<p>In no particular order...</p>
<ul>
<li><b><a href="http://seventhings.liftweb.net/security">Security</a>.</b> Big security focus. Immune to lots of common vulnerabilities. I think it even automatically uses random param names for POSTs, etc.</li>
<li><b>Speed.</b> Compiles to Java bytecode, runs on JVM. Parallel rendering. Scala has built-in, native XML support so that should be faster than parsing textual templates. I read somewhere that a modest, old processor can comfortably serve 300 req/sec on a single processor, doing a modest amount of transformation. With threading I've read that this can exceed 20x the speed of the same webapp in RoR running multi-process, presumably on something like Puma, Thin, whatnot. <em>(No sources, sorry, it's arbitrary anyway. And we all know Ruby can be scaled, that's not the topic here.)</em></li>
<li><b>Snippets rather than MVC.</b> MVC has never felt right to me. It's great for trivial DB-interface-like-webapp kind of stuff, but outside of that I've often run into ambiguous scenarios that don't feel comfortable... although I have used it happily for lack of a better alternative. Lift instead uses snippets which are pieces of logic/functionality that you can use all over the place in as many views as you like. Seems much better in terms of reusability, organisation on the other hand I'm not sure yet. Snippets are also executed in parallel -- nice.</li>
<li><b>Easier Ajax with <a href="http://seventhings.liftweb.net/wiring">wiring/binding</a>, <a href="http://seventhings.liftweb.net/comet">server-push</a>, more.</b> In addition to what the links say, most of the Ajax plumbing seems to be automatic; you don't even need to declare URLs or actions. Also type and parameter safety -- excellent.</li>
</ul>
<p>Those are the main reasons that come to mind. There other niceties too such as <a href="http://seventhings.liftweb.net/lazy">lazy loading</a>. The doco flaunts <a href="http://seventhings.liftweb.net/templates">designer friendly templates</a> as some awesome feature but I'm personally on the fence about it. I'm a one-man everything team at the moment so it doesn't immediately appeal to my situation. I can see it potentially making CSS dev faster (because you can work off a static template with all cases hardcoded which gets wiped by Lift) but I think any small gains will be offset by the major productivity loss of not having HAML.
</p>
<h1>Downsides. The Oh Noes.</h1>
<p>
Basically, less <good thing>.
</p>
<p>
Less adoption. Less incumbency. This results in less libraries to choose from. Less plugins. I assume more reinvent-the-wheel kind of stuff for me to write.
</p>
<p>
Less community. It especially won't be as big as RoR's. This means less examples and code online, less questions on StackOverflow, less forums and blogs, less information and help. It sounds like the mailing list is friendly enough and I'll be joining today but it's nicer to have more resources at your disposal rather than just always hitting up the same guys for help.
</p>
<p>
Less developers on the market. If this app makes me millions of dollars and I get to a point where I need to outsource or hire then I'm going have significantly less people that can do the job. It's hard not to compare to RoR which is ubiquitous these days and which would cause no sweat finding capable helpers.
</p>
<p>
I'm going to accept these problems because Scala is awesome, Lift is philosophically fresh and whispers of great advantages once you climb the learning curve, and there are aspects of Lift that will have a direct impact on time, money and resources. For example, I'll be able to host my project free for longer (due to improved performance). If my project gets popular, yes, I'll have it harder looking for manpower but, (and this runs contrary to my innate tendencies), I often read & hear (with supporting evidence) that it's best to focus entirely on the short term when starting up a small business or venture (notice I avoided calling it a "startup"). Worries like scalability, resourcing, support, etc. can and should be dealt with once the project premise is proven to be successful <b>and profitable</b>. Like the 37-Signals guys say, <em style="color:#008">"A business without a path to profit isn't a business, it's a hobby"</em>, and if this gets off the ground and becomes just a hobby then I'm not going to fuss about that stuff anyway.
</p>David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com2tag:blogger.com,1999:blog-5837865295676240265.post-80863930947506582682013-04-17T14:55:00.000+10:002013-04-17T14:59:01.396+10:00Lessons Learnt on My Second Android App - Pt.1<img style="width:30%; float:right; margin:0 0 12px 12px" src="http://i.imgur.com/3izJQ1Rl.png" title="Hosted by imgur.com" alt="" />
<p>
Hi. Recently I released my second Android app. It's a collection of game timers and bells and whistles to be used whilst playing board games, card games, party games, the like. Practically, that means you can use it to play 4 player Scrabble where each player has 2 minutes-per-turn and a total of 25 min for the entire game, for example. Or, for a game like Scattegories you can have an hourglass set to run for a random amount of time between 1~2 min and then scare the crap out of everyone when it goes off. That kind of thing.
</p>
<p>
If you're interested, you can peruse <a href="http://imgur.com/a/lF9dB">some screenshots here</a>
and find the app here: <a href="https://play.google.com/store/apps/details?id=com.beardedlogic.gametimers">“Time Us!”</a>
</p>
<p>
Moving on, I'd like to share and record for future-me, the lessons I learned and observations I made whilst creating this app.
</p>
<h1 style="clear:both">Scale Bitmaps Effeciently</h1>
<p>
Those tiny graphic files in your resources consume a surprising amount of memory when in use. A 100KB PNG is compressed and when loaded can comsume 10MB of memory as a Bitmap. As we know memory is quite important on mobile devices, especially on older devices. If the user is on a small LDPI device for example, their tired old phone isn't going to have much memory to spare. If you're scaling bitmaps there's a chance that you're consuming more memory that you need to, and the amount of waste gets higher with older devices where it matters more.
</p>
<p>
Say you have a large graphic that you scale to a smaller size, you're going to find that without some special options in-place, the device will load the full graphic into memory and retain it as it was loaded prior to scaling. I originally assumed that on a small LDPI device the graphic would scale down to about 20% (or whatever) of its size and accordingly only consume 20% of the full size in memory. Right? No. By default Android loads first, scales later and doesn't let go of that full-sized bitmap.
</p>
<p>
So how can you reduce memory consumption? The first recommended solution (because it is more processor-performant as well) is to prepare different copies of your graphics for each density. Ok great, but there are scenarios where that's not the best approach so what then? It turns out that there's a whole trove of info on this issue online in the official Android docs, the specifics on this scaling issue here:</br>
<a style="margin-left:20px" href="http://developer.android.com/training/displaying-bitmaps/load-bitmap.html">http://developer.android.com/training/displaying-bitmaps/load-bitmap.html</a>
</p>
<p>
You can mostly just copy-and-paste the utility code from the link above. By applying the methods therein, I was able to reduce memory usage by up to 80% or so (from memory), depending on the device. I didn't know this page existed until I needed it so I suggest you bookmark it and/or keep a mental note if you don't read the above page immediately.
</p>
<p>
<h1>Free Bitmap Memory Manually</h1>
<p>
Now that we know a little more about bitmap memory consumption, let's talk about another snag I encountered.
</p>
<p>
My app's home screen has two looks: dark & light.
The light-mode background consists of 2 images. On a w360dp XHDPI phone, it consumes 5.8MB for bkgd img #1, and 4.0MB for #2, making a total of 9.8MB.
</p>
<p><em>(FYI: You can easily see the memory profile of, and analyse your app by doing this from Eclipse: DDMS → Devices → Select process and click “Dump HPROF file” in the toolbar.)</em></p>
<p>
What would you expect to happen to that 9.8MB when the user hits a button and the app moves on to the next activity (screen)? I expected that the OS would release the memory because it's not in use. Not so. As long as the home activity is in the task history the memory will stay consumed and locked which in my case means for the entirety of the app. Now maybe technology is catching up but the old phone I had for two years before my newer Galaxy Nexus, was a HTC Magic, and that thing was constantly chugging due to memory problems and as such. Consequentially, I'm <em>not</em> comfortable with holding onto 10MB for no reason on a mobile device.
</p>
<p>
So what can you do? Well I could draw both images onto a single canvas but that's not really solving the problem, that would just drop the 10MB down to 6MB and it would still be unavailable later. To solve the problem I did three things:
</p>
<ol><li>
<strong>Remove, dereference, recycle the images in <code>onStop()</code>.</strong>
<br/>I wrote a utility function to do just that, and then simply called in in my activity's <code>onStop()</code> method. It looks like this:
<script src="https://gist.github.com/japgolly/5401310.js"></script>
</li><li>
<strong>Set the images during <code>onStart()</code>.</strong>
<br/>In your activity, just use <code>View.setBackground(Drawable)</code> or <code>View.setBackgroundDrawable(Drawable)</code> in your <code>onStart()</code> callback to restore the images that you clear in <code>onStop()</code>.
</li><li>
<strong>Turn off hardware acceleration.</strong>
<br/>Devices running Android 4.2 require an additional kick in the pants to get them to let go of the memory. You'll want to turn hardware acceleration off or else you'll end up with instances of <code>GLES20DisplayList</code> in memory that consume exactly the same amount of space. Instructions to turn off hardware acceleration can be found here: <a href="http://developer.android.com/guide/topics/graphics/hardware-accel.html">http://developer.android.com/guide/topics/graphics/hardware-accel.html</a>
</li></ol>
<p>
There's actually a whole bunch of doco on bitmaps and memory on the official Android site so for more info, take a look at <a href="http://developer.android.com/training/displaying-bitmaps/index.html">http://developer.android.com/training/displaying-bitmaps/index.html</a>.
</p>
<h1>PNGOUT</h1>
<p>
If you have a moderate amount of graphics and like me you optimise (ie. re-generate) them for multiple densities, then the size of your APK can balloon fast. This situation lead me to discover a utility called PNGOUT (<a href="http://advsys.net/ken/utils.htm">Windows</a>, <a href="https://aur.archlinux.org/packages/pngout/">ArchLinux</a>, <a href="http://www.jonof.id.au/kenutils">Other Linux + OSX</a>). It optimises your PNGs and gets them down to a smaller size, from memory, using a special compression algorithm targeted at graphics. As with most types of ultra-compression it can be slow but the results are worth it.
</p>
<p>
I ran it over my PNGs. It took 30min and reduced my total size from 7.4MB to 5.9MB, a 20% reduction. Nice.
</p>
<h1>Screen Widths</h1>
<p>
I learned a few things about screen width. Firstly I had to find a database of devices and specs. One doesn't exist but I did find two admirable attempts <a href="https://github.com/mataanin/android-devices">here</a> and <a href="http://checkscreensize.appspot.com/listdevice.jsp">here</a>. After some analysis, from what I can tell, most older phones have a width of 320dp and I don't think there are any phones with less than that -- AdMob's smallest banner ads are 320dp. For LDPI and MDPI, 320dp is a safe bet. Once you get to HDPI, it seems that around 70% of devices are 320dp, 25% are 360dp and the rest are larger than that. Here's the most interesting piece of news: for XHDPI it seems that the 360dp is the minimum and majority. There doesn't seem to be any 320dp XHDPI devices in existence!
</p>
<p>
What does this mean? Why do I care? It means that on some devices in some scenarios, you're going to end up with more free space that you expected and a smaller perception. Instead a larger graphic is more appropriate and that's something to consider when generating graphics. You'll need to make a conscious decision about which dimension is more important to you for each graphic, and enlarge appropriately. If you want a graphic to be perceived as being the same size on a 320dp-width and a 360dp-width HDPI device, then you might want to generate another copy at 360 ÷ 320 = 1.125x the original size. If you're supporting tablets then it might pay to do something similar for the most common tablet sizes.
</p>
<p>
Here's what I ended up with:
<table style="border-collapse:collapse; margin-left:auto; margin-right:auto">
<tr><th style="border:solid 1px #111; padding:2px; text-align:center">Dir</th><th style="border:solid 1px #111; padding:2px; text-align:center">Factor</th><th style="border:solid 1px #111; padding:2px; text-align:center">Content</th></tr>
<tr><td style="border:solid 1px #111; padding:4px"><code>drawable-ldpi</code></td><td style="border:solid 1px #111; padding:4px">0.75</td><td style="border:solid 1px #111; padding:4px">Everything</td></tr>
<tr><td style="border:solid 1px #111; padding:4px"><code>drawable-mdpi</code></td><td style="border:solid 1px #111; padding:4px">1</td><td style="border:solid 1px #111; padding:4px">Everything</td></tr>
<tr><td style="border:solid 1px #111; padding:4px"><code>drawable-hdpi</code></td><td style="border:solid 1px #111; padding:4px">1.5</td><td style="border:solid 1px #111; padding:4px">Everything</td></tr>
<tr><td style="border:solid 1px #111; padding:4px"><code>drawable-w360dp-hdpi</code></td><td style="border:solid 1px #111; padding:4px">1.6875</td><td style="border:solid 1px #111; padding:4px">Width-sensitive GFX only</td></tr>
<tr><td style="border:solid 1px #111; padding:4px"><code>drawable-xhdpi</code></td><td style="border:solid 1px #111; padding:4px">2 or 2.25 depending on the gfx</td><td style="border:solid 1px #111; padding:4px">Everything</td></tr>
</table>
</p>
<p>
If you'd like to learn more have a read of these:
<br/><a style="margin-left:20px" href="http://developer.android.com/training/multiscreen/screendensities.html">Supporting Different Densities</a>
<br/><a style="margin-left:20px" href="http://developer.android.com/training/multiscreen/screensizes.html">Supporting Different Screen Sizes</a>
</p>
<h1>More Next Time</h1>
<p>
I learnt more but I'll post that next time. If you read this far I hope I've been helpful.
</p>David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com1tag:blogger.com,1999:blog-5837865295676240265.post-21928530656168493372013-02-19T08:54:00.000+11:002013-02-19T09:12:26.956+11:00Android Project Templates<p>
Today I'd like to share some templates I've made for creating Android apps. They all use Maven and have been coerced into compatibility with Eclipse. There are a few different flavours which I will describe below. If you want to skip the reading and just want the code, np, it's all here: <a href="https://github.com/japgolly/reference">https://github.com/japgolly/reference</a>.
</p>
<h1>Template #1: Java</h1>
<p><a href="https://github.com/japgolly/reference/tree/master/android-java-maven-single">Code available here</a>.</p>
<p>
This is the gist of what I use when I'm using Android + Java. It comes with a few add-ons to mitigate Java's verbosity and inflexibility and the fact that they all work together in harmony, even with Eclipse, well that's what makes this template gold. It took a while to get working way back when.
</p>
<h5>Features</h5>
<ul>
<li><strong>Lombok</strong>: Integrated into both Maven and Eclipse.<br/>Modified to play nice with Android, and added the ability to add <code>@Inject</code> to generated constructors.<br/><em><a href="http://projectlombok.org/">What is Lombok?</a></em></li>
<li><strong>AndroidAnnotations</strong>: Integrated into both Maven and Eclipse.<br/><em><a href="http://androidannotations.org/">What is AndroidAnnotations?</a></em></li>
<li><strong>CoFoJa</strong>: Contracts For Java provides annotations that can be used to verify pre-conditions, post-conditions and invariants. The validations are inherited so they're great for interfaces. Also they can be omitted from bytecode during compilation meaning you get more checking during tests and dev with no performance penalty in prod.<br/><em><a href="http://code.google.com/p/cofoja/">More about CoFoJa.</a></em></li>
<li><strong>Unit testing</strong>: Robolectric is being used for unit testing (along with an example of usage). I'm actually using a <a href="https://github.com/japgolly/android-test-utils">test library of mine</a> (open-source) that includes Robolectric, FEST, Mockito and some Android-specific testing utilities.<br/><em><a href="http://pivotal.github.com/robolectric/">What is Robolectric?</a></em></li>
<li><strong>Integration testing</strong>: Robotium is being used for integration testing (along with an example of usage).<br/><em><a href="http://code.google.com/p/robotium/">What is Robotium?</a></em></li>
<li><strong>Android Lint</strong>: There's a profile that run Android Lint. Great for CI builds.</li>
<li><strong>Findbugs</strong>: Findbugs has been integrated (not that it was effort). What's nice is that I've included a rules file to exclude certain Android-specific warnings. I'm a nice guy.</li>
<li><strong>Proguard</strong>: Dev & release configs for Proguard are included.</li>
<li><strong>Release</strong>: A release profile is included that runs Proguard, signs artifacts, and runs zipAlign.</li>
<li><strong>AdMob</strong>: It's in there, ready to go but can also be deleted just as easily if you don't want it.</li>
<li><strong>Eclipse settings</strong>: Auto-format on save. Auto-organise-imports on save. A <code>TODOC</code> tag for doco todos. More.</li>
</ul>
<h1>Template #2: Scala</h1>
<p><a href="https://github.com/japgolly/reference/tree/master/android-scala-maven-single">Code available here</a>.</p>
<p>
Here you can use Scala instead of Java to build your app. The introduction of Scala renders obsolete the need for Lombok and AndroidAnnotations which simplifies the build process. As above, everything in this build works from CLI with Maven, and within Eclipse.
</p>
<h5>Features</h5>
<ul>
<li><strong>Scala</strong>: Scala app code. Scala unit tests. Scala integration tests. Scala proguard. Scala Eclipse. Scala everything!</li>
<li><strong>Unit testing</strong>: Robolectric is being used for unit testing (along with an example of usage). I'm actually using a <a href="https://github.com/japgolly/android-test-utils">test library of mine</a> (open-source) that includes Robolectric, FEST, Mockito and some Android-specific testing utilities.<br/><em><a href="http://pivotal.github.com/robolectric/">What is Robolectric?</a></em></li>
<li><strong>Integration testing</strong>: Robotium is being used for integration testing (along with an example of usage).<br/><em><a href="http://code.google.com/p/robotium/">What is Robotium?</a></em></li>
<li><strong>Android Lint</strong>: There's a profile that run Android Lint. Great for CI builds.</li>
<li><strong>Proguard</strong>: Proguard is mandatory when Scala is used. Rules included so that Scala works. Dev & release configs for Proguard are included.</li>
<li><strong>Release</strong>: A release profile is included that runs Proguard, signs artifacts, and runs zipAlign.</li>
<li><strong>AdMob</strong>: It's in there, ready to go but can also be deleted just as easily if you don't want it.</li>
</ul>
<p>NOTE: You cannot use Scala <em>and</em> Java together because it wrecks Eclipse. If you're using VIM or some other editor inferior to mighty VIM, you'll be fine with both languages simultaneously, Maven can handle it, just don't expect a happy life using Eclipse.</p>
<h1>Template #3: Scala + Free/Paid Editions of Your App</h1>
<p><a href="https://github.com/japgolly/reference/tree/master/android-scala-maven-dual">Code available here</a>.</p>
<p>
Same as the Scala template above except this time instead of having a single app, you have an Android library and two separate apps that extend it.
</p>
<h5>Features</h5>
<ul>
<li><strong>Everything in template #2</strong>. See above.</li>
<li><strong>Shared APK Library</strong>: Anything shared between your free & paid apps will live in the shared library. It's an Android library so resources can be shared too.</li>
<li><strong>Separate Apps</strong>: Your free and paid apps are separate apps that extend the shared library. Additional code and/or resources can be dropped in as necessary.</li>
</ul>
<h1>Usage</h1>
<p>To use one of these templates just copy the entire folder (and ensure you capture files starting with <code>.</code>). Next use the <a href="https://github.com/japgolly/reference/blob/master/find_template_values.sh">find_template_values.sh</a> script to find places in the project where you need to swap out example values for real values. There's also a helper script, <a href="https://github.com/japgolly/reference/blob/master/replace.sh">replace.sh</a> that you can use to perform recursive, mass regex replacements.
</p>
<p>To load your project into Eclipse, simply hit <em>File</em> → <em>Import...</em> → <em>Existing Projects into Workspace</em>. All the required Eclipse settings and files are already there in the template.
<p>
<p>Enjoy.</p>David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com0tag:blogger.com,1999:blog-5837865295676240265.post-23295248383645236152013-02-10T09:44:00.001+11:002013-02-10T09:50:52.684+11:00Releasing an App for Android - Part 2<p>
Continuing on from <a href="http://japgolly.blogspot.com.au/2013/02/releasing-app-for-android-part-1.html"> part 1</a>...
</p>
<h1>4. Free and Paid Editions.</h1>
<p>
The Google Play marketplace doesn't allow free & paid editions of apps. If you want free & paid editions you will need to manage and release two separate apps with separate package names.
</p>
<p>
To do this I decided to go with the mainstream approach, namely, creating a separate projects. If you have shared code without shared Android resources (unlikely) then you'd simply move the shared code out into its own project and include the resulting jar as a dependency in the free & paid projects. With Android it's similar, you create an APK library. Practically this means adding <code>android.library=true</code> to your library's project properties and changing its pom packaging from <code>apk</code> to <code>apklib</code>. The free/paid projects wont contain much but they will need their own AndroidManifests and a reference to the library in the project properties such as <code>android.library.reference.1=../library</code>. That's the gist of it.
</p>
<p>
One annoying thing is that, like a few things that have come out of the Android dev camp, it's a hack. The resulting library is basically just a zip of the resources and the <em>SOURCE CODE</em>. When you use an Android library in a top-level project it unzips, merges and recompiles the library code. How annoying. The reason is the generated <code>R.java</code> will produce different ids outside of the library, hence the recompilation. As far as I understand the system is therefore a quick hack. It might be fair enough - I don't know how much pressure the Android dudes are under or how well/poorly staffed they are. But technologically speaking is still a hack. Oh well. Make sure Scala incremental compilation is on as it takes a large bite of the build pain.
</p>
<p>
In retrospect it was a good thing that I saved this step until the end. The time to alter the build & structure (especially if you've got a working reference) near the end of the project will be much less than both the time spent by managing 3 separate projects instead of 1; and the time wasted by waiting on the extra compilation, dexing and proguard'ing.
</p>
<p><strong>Tip #1:</strong> Use a single project for as long as you can get away with but anticipate that towards the end you'll need a little time to split out library, free, and paid projects.
</p>
<p><strong>Tip #2:</strong> Anything edition-specific should be separated from anything shared. For example, create an ad banner layout and then <code><include></code> it rather than using an AdView directly. Later you can configure things so that the paid edition gets a NOP layout.
</p>
<h1>5. Advertising.</h1>
<p>
I started not knowing anything about ads. Basically I did this:
<ol>
<li>Sign up with AdMob.</li>
<li>Fill in all your bank details so they can pay you. You'll also need a merchant account with Google Play.</li>
<li>Add your app but don't give it a market URL yet. This allows you to get an ID that you can start using before you release.</li>
<li>AdMob doesn't just show you their own ads; they connect to other ad networks and can show you their ads too. That was surprising. Advantage us because it means that once you get ads working in your app and you want to use a different ad provider like InMobi or MobFox, you can just sign up with those guys, get your ID and plug it into AdMob on their website. No code changes required. You can even configure Admob to show ads from a combination of providers, or use different providers for certain countries.</li>
<li>Get the AdMob SDK and wire it into your project.</li>
<li>Follow the instructions on this page to integrate ads into your layout: <a href="https://developers.google.com/mobile-ads-sdk/docs/admob/fundamentals">https://developers.google.com/mobile-ads-sdk/docs/admob/fundamentals</a>. Reading all those pages is a good idea.</li>
<li>Finally, if you want ads to refresh every 60 sec or so, don't do it in code; play around in the AdMob website and you'll find settings for it. You can also customise the appearance of ads on the website or in code.</li>
</ol>
Done.
</p>
<p>
Now I don't know much about how it all works yet but this is what I've gleamed:
<ul>
<li>Every time an ad is shown to a user it's called an <em>“impression”</em>. You get nothing, $0.00, for impressions.</li>
<li>Every time an ad is clicked you get something small like $0.03.</li>
<li>RPM = <em>“Revenue Per Mile”</em> = the amount of money you've made per 1000 impressions.</li>
<li>Fill rate = Quote: <em>Fill rate represents the percentage of ad requests that satisfy the ad requests sent by the app. It is a measure of AdMob's ability to serve ads in your app with the existing inventory.</em></li>
<li>You'll see eCPM everywhere, it stands for <em>“effective cost per mile”</em>. It's a metric indicating how effective/profitable a particular ad network has been for you <em>so far</em>. Don't freak out when AdMob says eCPM $0.00 when you first sign up, that just means you haven't had any clicks yet.
The formula is <code>1000 x Cost-per-Click x Click-Through-Rate.</code></li>
</ul>
</p>
<p>And we've reached the boundary of my knowledge on the topic. Hope you enjoyed the tour.</p>
<p><strong>Tip #1:</strong> Filter logcat by tag <code>“Ads”</code> to see all your AdMob logs.
</p>
<p><strong>Tip #2:</strong> Search the AdMob logs for your test device ID and then plug that into the <code>ads:testDevices</code> attribute of your view. From what I hear the AdMob support is notoriously bad so if you get in trouble with your account, it's far from easy to get it back online.
</p>
<p><strong>Tip #3:</strong> eCPM is a metric of how well an ad network has been for someone. It's not a setting that you need to configure (even though it looks that way from AdMobs UI).
</p>
<h1>6. Localisation.</h1>
<p>
Not much to say here. Android warns you constantly about externalising your strings which is good. It doesn't monitor your source code though. If paranoid or you have a large team, Eclipse can warn when it finds strings without something like <code>//$NON-NLS-1</code>. Checkstyle et al probably have a similar feature.
</p>
<p><strong>Tip #1:</strong> Either put non-translatable strings into a file called <code>donottranslate.xml</code> (seriously – Android Lint requirement) or give them an attribute <code>translatable="false"</code>.
</p>
<p><strong>Tip #2:</strong> A split to paid/free editions will introduce new strings. Consider this before sending out text for translation.
</p>
<p><strong>Tip #3:</strong> The marketplace description(s) will also need translation. Write and include that before sending out text for translation.
</p>
<p><strong>Tip #4:</strong> Your text is easier to manage when all in one place. Therefore don't include separate strings in the free/paid versions; put them all in the library project together then just reference them differently in each app. If you don't want to change the references then reference an alias and change the alias in each project. Example:
<blockquote><code><pre>
Library project:
<string name="app_name_free">Bananas (free)</string>
<string name="app_name_paid">Bananas (pro)</string>
<string name="app_name">@string/app_name_paid</string>
Free project:
<string name="app_name">@string/app_name_free</string>
</pre></code></blockquote>
</p>
<p><strong>Tip #5:</strong> Use the app in each language to spot-check the layout. Certain languages might need tweaking to look good. For example, a language like Japanese that doesn't use spaces might need a manual endline (<code>\n</code>) so that it doesn't end up with one character dangling and the word split across lines.
</p>
<h1>7. Proguard.</h1>
<p>
If you don't know what Proguard is, it's a tool that shrinks, "validates", optimises and obfuscates your binaries.
</p>
<p>
Scala on Android demands it.
<br/>Optimisation is very slow.
<br/>Obfuscation didn't seem to do very much to my app although it happily obfuscated my dependency libraries.
</p>
<p>
I managed to speed things up by having two proguard configs: dev & release. Turn off optimisation and obfuscation in the dev config and you will save a lot of time. With optimisation enabled it can take minutes at a time. I've uploaded my configs here: <a href="https://gist.github.com/japgolly/4747423">https://gist.github.com/japgolly/4747423</a>
</p>
<p><strong>Tip #1:</strong> Reuse existing proguard configs.
</p>
<p><strong>Tip #2:</strong> Use separate dev & release proguard configs.
</p>
<h1>8. Signing.</h1>
<p>
Your app will need to be signed (and then zip-aligned, in that order) in order to release it.
To sign it you need to have your own certificate that doesn't expire until at least 2033 (which isn't the best for security but I don't make the rules). It's a Java thing so just google <code>keytool</code> and you'll be done pretty quickly. You'll need to create and save 2 passwords: one for the key, one for the store. I generated mine with this command: <em>(tip: script it or purge it from shell history when done)</em>
<blockquote><code><pre>
keytool -genkey -v \
-keystore <keystore filename> \
-alias <keystore name> \
-keyalg RSA -keysize 4096 -validity 10000 \
-keypass 'xxxxxxxxxx' \
-storepass 'yyyyyyyyyy'
</pre></code></blockquote>
</p>
<p>Once you're done, make sure you backup your keys and passwords then it's time to integrate signing into your build. For a Maven project you need to weave a massive blob of shit into your poms in order to integrate signing and zip-align. They should be in a release profile and all up it comes to around 100 lines of XML (!). I don't remember how much I was able to refactor into my parent pom (as my project is multi-module) but I don't think it was all of it. I'll post all the Maven stuff shortly anyway. Most of the time spent on signing was getting Maven to work properly.
</p>
<p><strong>Tip #1:</strong> Either ditch Maven for something better, or make sure you copy a working Maven Android project that has signing. (I'll post my Maven stuff shortly.)
</p>
<h1>9. Build Automation.</h1>
<p>
Last but not least, we arrive at something that is actually important from day #1 of the coding period: build automation. Very important. Obviously. I used Maven and ended up creating a multi-module project which I'll post more on later. Suffice to say Maven gave me trouble, and cost me too much time. I want to switch to something else and I have my eye on <a href="http://www.scala-sbt.org/">SBT</a>.
</p>
<p>I did some tests on SBT recently and the results supported the rumours that SBT is faster than Maven. A build of a single, simple Android project took 35 sec with Maven, verses the same project at 24 sec with SBT. That's a 31% saving. A clean build of my multi-module Maven project takes 2 minutes and I wonder how fast it would be with SBT. If I assumed flat 31% saving again, that's 120 sec down to 82 sec. Nice. I didn't investigate further because it's time-consuming and because I worry about Eclipse integration. I'm sure Eclipse + SBT = happy days, but it seems that Eclipse + ADT + SBT = pain. ADT is extremely inflexible and the sbt-android plugin is way too poorly documented for an SBT noob like me. If I changed the dir & file structure to suit ADT an expert might be able to modify the sbt-android build pretty easily but I can't. Also, it practically has a seizure if you give it a full AndroidManifest as it wants to create its own. I imagine that ADT wouldn't be too happy about that. Would loooooove to be shown the error of my ignorant ways here so if anyone knows, ping me.
</p>
<p>
Another thing is that Maven is archaic and verbose. Reuse between modules is impossible in some situations; there are 6-year-old bugs and feature requests for mixins that are still open... SBT on the other hand is concise and allows as much reuse as you can shake a stick at (and I can shake a stick at A LOT of reuse).
</p>
<p>
But any further bitching about Maven is just that at this point: bitching. So I'll suck it up and declare that though sweat and toil I've come to have a great multi-module Maven Android project setup with library, free/paid projects, and free/paid (instrumented) integration test projects working together, nuances coerced, effective for dev and release builds. I'll be posting it shortly.
</p>
<h1>In Conclusion</h1>
Looking back on all these issues I can now see how it took 5 or 6 days to go from 95% dev complete to released, despite doing 0%-95% in 2 days. I'll be applying these lessons during my other Android projects and I hope it helps others <em>(yes you!)</em> as well.David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com1tag:blogger.com,1999:blog-5837865295676240265.post-80755110521412014942013-02-08T12:01:00.002+11:002013-02-10T09:45:41.382+11:00Releasing an App for Android - Part 1<p>
Last week I decided I would create a little micro Android app and release it so that I could get experience with the process. I recently came to the opinion that it's a good idea to hit learning curves with something small first rather than attempting both “big” and “new” at the same time. So I created an Android app called <a href="https://play.google.com/store/apps/details?id=com.beardedlogic.bpm.free" target="_blank">BPM Tapper</a>.
What I found surprising was that I went from vague idea to 95% code complete in 2 days, but then it took another 5 or 6 days to get it released.
</p>
<p>
Now, my goal here with this blog post is twofold. Firstly, to document what I went through releasing my first Android application so that in future I remember everything that needs to be done; great for planning. Secondly, I want to document tips and strategies to improve next time.
</p>
<br/>
<h1>1. Android Completeness</h1>
<p>
By <em>Android Completeness</em> I'm referring to things that are either required or expected of common Android applications. The tasks I came across are as follows.
</p>
<h2>1.1. Inspecting Android Lint warnings.</h2>
<p>This is an easy one. The Android SDK provides a lint tool that analyses your compiled code and resources, and creates a list of warnings. Not all Lint warnings are correct but they all should be vetted and resolved if necessary.
</p>
<h2>1.2. Handling interruption.</h2>
<p>Your app can be interrupted at any time, e.g. if the phone gets a call while your app is in use, the OS will kick your app out and data will be lost if you don't handle it. Therefore if you don't want users to lose their on-screen state when using your app (or changing orientations), you generally need to implement two methods in your activities. <code>onRestoreInstanceState</code> and <code>onSaveInstanceState</code>. Scala made this really simple; case classes are serialisable so one simply calls <code>Bundle.[get|put]Serializable</code>. Example:
<blockquote><code>
val measure = savedInstanceState .getSerializable(BUNDLE_KEY_MEASURE).asInstanceOf[Measure]
</code></blockquote>
Too easy.
</p>
<h2>1.3. Integrating with the <a href="http://developer.android.com/guide/topics/data/backup.html" target="_blank">Android Backup Manager</a>.</h2>
<p>My app is stateless so I just flipped the <code>allowBackup</code> switch off in the <code>AndroidManifest</code> and didn't have the need to write any code here, but it is something to be aware of. It's easy to forget but very important for stateful apps.
</p>
<h2>1.4. Supporting multiple screen densities.</h2>
<p>Different phones have different screen densities and you should at least be targeting support for MDPI through to XHDPI. Different screen densities (loosely) require different copies of your graphics at different sizes. This wasn't a problem for me because I create all my graphics in SVG and have a script to convert them to PNGs using ImageMagick. (See the Artwork section for such a script.)
</p>
<h2>1.5. Providing landscape layouts.</h2>
<p>Not mandatory but I've seen quite a few reviews on the marketplace where people get quite passionate (read: vehement) about not being able to turn their phone on its side. Personally I don't see why it's such a big deal but I'm not everyone. Creating a new layout for landscape-mode and having it coexist is easy. However, <em>designing</em> the landscape UI that is pleasing and effective is the time-consuming part.
</p>
<p><strong>Tip #1:</strong> Always use something like Inkscape to create the screen before you hit the code. Use real sizes and colours. Don't assume and leave out anything that you plan to do in code later.
</p>
<p><strong>Tip #2:</strong> Commit to doing this to all screens in your app up-front, or no none at all. Or do some. But make that decision early on and ensure you can make it work. You don't want to create 8 landscape layouts then later come across a few that won't work then decide to scrap the entire landscape orientation ability.
</p>
<h2>1.6. Ensuring layouts look acceptable in various conditions.</h2>
<p>This means compiling a testing matrix of relevant attributes such as:
<ul>
<li>Screen size.</li>
<li>Screen density.</li>
<li>Screen orientation.</li>
<li>Android version.</li>
<li>Ads vs no-ads (they do take up valuable screen space).</li>
<li>Language.</li>
</ul>
This is pain. The theory is simple right? The screen will look different depending on each factor above. You want your beloved app to look great on all users' phones.
</p><p>
Screen density I've addressed above. That, language and Android API version can just be eyeballed manually I think. No need to give them their own axis in a matrix. But I did want a matrix of ads-or-not, orientation, and screen size. What was annoying is that screen size is not screen size. There are resource qualifiers for screen size, namely <code>small</code>, <code>normal</code>, <code>large</code>, <code>xlarge</code>. Now they didn't work as expected for me. Due to my immense frustration and exhaustion at the time, I don't remember the details but I found them not aligning with my expectations at all. Instead I decided to use screen density to create different settings for different screen sizes, because there is a strong correlation between density and screen size anyway. I mean, no one has a 1200x800 LDPI phone. It doesn't exist. Thus I used a test matrix like this:
</p>
<table style="border-collapse:collapse; margin-left:auto; margin-right:auto">
<tr><th></th><th style="border:solid 1px #111; padding:2px; text-align:center" colspan="2">ads</th><th style="border:solid 1px #111; padding:2px; text-align:center" colspan="2">no ads</th></tr>
<tr><th></th><th style="border:solid 1px #111; padding:2px; text-align:center">port</th><th style="border:solid 1px #111; padding:2px; text-align:center">land</th><th style="border:solid 1px #111; padding:2px; text-align:center">port</th><th style="border:solid 1px #111; padding:2px; text-align:center">land</th></tr>
<tr><th style="border:solid 1px #111; padding:2px">ldpi</th><td style="border:solid 1px #111; padding:2px; text-align:center">✓</td><td style="border:solid 1px #111; padding:2px; text-align:center">✓</td><td style="border:solid 1px #111; padding:2px; text-align:center">✓</td><td style="border:solid 1px #111; padding:2px; text-align:center">✓</td></tr>
<tr><th style="border:solid 1px #111; padding:2px">mdpi</th><td style="border:solid 1px #111; padding:2px; text-align:center">✓</td><td style="border:solid 1px #111; padding:2px; text-align:center">✓</td><td style="border:solid 1px #111; padding:2px; text-align:center">✓</td><td style="border:solid 1px #111; padding:2px; text-align:center">✓</td></tr>
<tr><th style="border:solid 1px #111; padding:2px">hdpi</th><td style="border:solid 1px #111; padding:2px; text-align:center">✓</td><td style="border:solid 1px #111; padding:2px; text-align:center">✓</td><td style="border:solid 1px #111; padding:2px; text-align:center">✓</td><td style="border:solid 1px #111; padding:2px; text-align:center">✓</td></tr>
<tr><th style="border:solid 1px #111; padding:2px">xdpi</th><td style="border:solid 1px #111; padding:2px; text-align:center">✓</td><td style="border:solid 1px #111; padding:2px; text-align:center">✓</td><td style="border:solid 1px #111; padding:2px; text-align:center">✓</td><td style="border:solid 1px #111; padding:2px; text-align:center">✓</td></tr>
</table>
<p><strong>Tip #1:</strong> Create a test matrix and use it to ensure things look good.
</p>
<p><strong>Tip #2:</strong> Eyeball the smallest screen sizes once in each language. Button widths can be twice as long as English depending on the target language which may cause side-effects with your layouts.
</p>
<p><strong>Tip #3:</strong> Always start with the lowest density and work your way up. A MDPI screen will choose <code>values-ldpi/</code> over <code>values/</code> if it exists and <code>values-mdpi/</code> doesn't.
</p>
<p><strong>Tip #4:</strong> Don't trust the screen heights you can choose in the ADT layout editor. I have an old HTC Magic that has around 100dp less height than the closest config available from the editor.
</p>
<p><strong>Tip #5:</strong> Do not leave any hardcoded dimensions in your layout XML (<em>obvious</em>) or your <code>styles.xml</code> (<em>not obvious</em>). When adjusting for each screen it's going to be the dimensions like the text sizes, the padding & margin sizes that you'll be adjusting.
</p>
<p><strong>Tip #6:</strong> Do your utmost, do your friend's utmost (!) to avoid using dimensions to affect the layout itself. For example, a dimension for the margin between elements is one thing but if you want a large river of negative space, don't use dimensions, anchor both sizes of the space to stuff so that the river size is organic. This makes for more consistent layouts on different devices and means you won't have to adjust the value for different screens.
</p>
<h1>2. Scala.</h1>
<p>Learn it! Use it! It is a brilliant language. It allows you to write better code, more concisely. There is so much boilerplate bullshit that Java requires, so many hoops to jump through, and so many obstacles that prevent you writing the kind of code that you (well, I) want to. Scala does away with nearly all of those! Its strong focus on immutability, functions, maximum code reuse (yay for multiple inheritance again!) and conciseness, not to mention all the other modern goodness such as pattern matching/partial functions; for comprehensions; case classes; implicit conversions, classes, and parameters... The list goes on. It's very productive language and you will be more productive with it.
</p><p>
Like anything it does have it's shortcomings though: flaky IDE support, slow compilation time, WTF-inducing stacktraces, controversial lack of binary compatibility between major versions (which doesn't bother me at all actually), massive stdlib that Android can't handle without proguard, too many custom operators at times (look at SBT with its <code><+= <<= ++= <++=</code> etc, good god!).
</p><p>
But all in all, Scala is worth it. IDE support will continue to improve and compilation time and stdlib modularisation should be fixed later this year.
</p><p>
Scala can especially be a godsend when it comes to cutting out Android/Java repetition. Consider this Java:
<blockquote><code><pre style="color:#511">
protected TextView nameView;
protected void onCreate(Bundle savedInstanceState) {
...
nameView = (TextView) findViewById(R.id.name);
}
</pre></code></blockquote>
And now consider this Scala:
<blockquote><code style="color:#151"> lazy val nameView = find[TextView](R.id.name) </code></blockquote>
Here's another example. Java:
<blockquote><code><pre style="color:#511">
view.setOnClickListener(new OnClickListener() {
@Override
public void onClick(View view) {
onDelete();
}
});
</pre></code></blockquote>
Verses Scala:
<blockquote><code style="color:#151"> view.setOnClickListener( onDelete _ ) </code></blockquote>
</p><p>
Scala also saved me time on the testing front. Your test code becomes amazing when you can use traits for multiple inheritance, and write methods that accept lambdas or Ruby-style blocks. You'll end up with something approaching a testing DSL before you know it, and most of your test cases will be under 3 lines which certainly taking the wind of the sails of any test-case-writing-laziness.
</p>
<p><strong>Tip #1:</strong> Use an incremental compiler for Scala. With the scala-maven-plugin this is a single config item.
</p>
<p><strong>Tip #2:</strong> Copy a proguard.cfg with Scala settings and set your build to always use proguard.
</p>
<p><strong>Tip #3:</strong> Use traits to create collections of test helper code. Do as little directly as possible in your immediate test case code.
</p>
<p><strong>Tip #4:</strong> Use implicit conversions for all things Android.
</p>
<p><strong>Tip #5:</strong> Create an ActivityHelperTrait or similar and load it up with implicit value classes and inlined helper methods. Example:
<blockquote><code> @inline def find[T <: View](resId: Int): T = findViewById(resId).asInstanceOf[T]
</code></blockquote>
</p>
<h1>3. Artwork.</h1>
<p>Get in the habit of using vector-based graphics and using ImageMagick's <code>convert</code> tool (it's command-line). Vector graphics look great at any size and you can <code>convert</code> to generate bitmaps at various sizes (which Android needs). If you want you can use SVGs directly in Android with <a href="https://github.com/japgolly/svg-android" target="_blank">svg-android</a> but there are still places where you need fixed-size images.</p>
<p>You will need to create or procure a visual to use as your application icon before release. Ensure you have it as an SVG (or some other vector-based format) then use ImageMagick to create various sizes. The marketplace requires 512x512, where as your app requires 36x36, 48x48, 72x72, 92x92 for ldpi, mdpi, hdpi and xhdpi respectivelly.
</p>
<p>For all kinds of visuals you'll need to have multiple copies in separate sizes to support different screen densities. I wrote a Ruby script to convert all my visuals. Here is an except of that script:
</p>
<p><script src="https://gist.github.com/japgolly/4735584.js"></script></p>
<p><strong>Tip #1:</strong> Use 32-bit PNGs if you want an alpha channel.
</p>
<p><strong>Tip #2:</strong> ImageMagick will be your best friend for conversion. Script it.
</p>
<p><strong>Tip #3:</strong> Inkscape is a great tool. Very easy to use and everything is vector-based.
</p>
<p><strong>Tip #4:</strong> There are plenty of good-looking, vector-based visuals out there that are public-domain. There are also sites like <a href="http://thenounproject.com/" target="_blank">The Noun Project</a> that have brilliant visuals that you can usually use free as long as you attribute the author, or can buy cheaply.
</p>
<br/>
<h1>To be continued...</h1>
I have 6 more points. Read about them <a href="http://japgolly.blogspot.com.au/2013/02/releasing-app-for-android-part-2.html">in part 2</a>.David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com7tag:blogger.com,1999:blog-5837865295676240265.post-30069490037759751092012-11-18T19:40:00.003+11:002012-11-18T19:47:39.760+11:00Maven: Now with Colour™ !<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-size: large;">M</span>aven.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I'm an command-line kinda guy. GUIs didn't event exist when I was a kid. When I use Maven it's often from the command-line. A curious thing happens when you run Maven from the CLI more than 3 times in a row. You slowly lose the ability to read those useful little symbols that we affectionately refer to as "the alphabet", your eyes begin to ache and you start to wonder who's really in control: you or your eyelids, the list oozes on. The reason for this is that you get a lot of information as the build runs, and more importantly it's all the same colour, usually bright white on black depending on your terminal config. Things get really hard to see and it takes a lot of precious concentration and brain-juice to decipher the results. Usually this is because you're searching for 1 particular line out of 100 or so with no real visual hints. I don't like it. My brain may well just be getting old or maybe I'm just working too much lately but I find that I don't really have much brainpower to spare, especially when you're in the middle of a problem with a large context. Don't like it.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-size: large;">S</span>o I wrote a script.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">Get it here: </span><a href="https://gist.github.com/4104053" style="font-family: Arial, Helvetica, sans-serif;">https://gist.github.com/4104053</a><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">The great thing about Linux is that you can treat nearly everything as either a file (like </span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">/dev</span><span style="font-family: Arial, Helvetica, sans-serif;"> and </span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">/proc</span><span style="font-family: Arial, Helvetica, sans-serif;">, awesome!) or a <b>pipeline</b>. By simply piping Maven output through a processor (while being careful not to block) you can apply colour all over the place by using some smart regex to insert strings that the terminal parses in order to allow user control over various terminal attributes, in this case: colour. </span><i style="font-family: Arial, Helvetica, sans-serif;">(<span style="font-size: x-small;">See <a href="http://en.wikipedia.org/wiki/ANSI_escape_code">ANSI escape codes</a> on Wikipedia for more info.</span>)</i><span style="font-family: Arial, Helvetica, sans-serif;"> Now I'm not the <a href="http://www.ensor.cc/2012/02/colorized-maven-output-on-mac-osx.html">first</a> <a href="https://github.com/builddoctor/maven-antsy-color">person</a> to <a href="https://gist.github.com/1027800">take</a> <a href="http://blog.blindgaenger.net/colorize_maven_output.html">this</a> approach, but the great thing is that because these scripts are so easy to write, anyone can create </span><span style="font-family: Arial, Helvetica, sans-serif;">or customise one </span><span style="font-family: Arial, Helvetica, sans-serif;">for their own personal needs.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">My script uses said approach to do the following:</span><br />
<ul style="text-align: left;">
<li><span style="font-family: Arial, Helvetica, sans-serif;">Clear the screen before running.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Highlight the name of each Maven phase and the plugin & goal responsible.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Colour the number of test passes, failures, errors.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Colour the test classes & methods of test failures and errors.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Colour Maven warnings & errors.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Highlight total build success / failure.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Remove that bunch of shit you get at the end of failed Maven builds. </span><span style="font-family: Arial, Helvetica, sans-serif;">When Maven fails (or tests fail) it dumps a bunch of info that basically amounts to "<i><span style="font-size: x-small;">Try using -e or -X else here's our website.</span></i>" Ok for beginners maybe but I don't need to see it all the time. I have "</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">mvn -help</span><span style="font-family: Arial, Helvetica, sans-serif;">" and Google if I need them.</span></li>
</ul>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">Now that feature-set might not sound like much but makes <b>a world</b> of difference. It turns my frown upside down. Twice! <i><span style="font-size: x-small;">(Hey...but that means-</span></i></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-size: large;">H</span>ere are some screenshots:</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div style="text-align: center;">
<span style="font-family: Arial, Helvetica, sans-serif;"><i>Coloured Maven when things are happy</i></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4nAU2dNj_OQ2FKQR-0PQInuI9p9ICGjAhXe2XL9Tu6mZM8hmv8Zva9DexoCyH9tiOrGX9ai38PDby-4A4ffAVuJ9Yfnz4m4DjWzlX5KLRViI0_0ZKG9jy4ut11OL-yuuXmsosr3PCP0uG/s1600/mvnc_ok.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="310" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4nAU2dNj_OQ2FKQR-0PQInuI9p9ICGjAhXe2XL9Tu6mZM8hmv8Zva9DexoCyH9tiOrGX9ai38PDby-4A4ffAVuJ9Yfnz4m4DjWzlX5KLRViI0_0ZKG9jy4ut11OL-yuuXmsosr3PCP0uG/s320/mvnc_ok.png" width="320" /></a></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div style="text-align: center;">
<span style="font-family: Arial, Helvetica, sans-serif;"><i>Coloured Maven when things go wrong</i></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiuCkxlVziRcFC-u_iLRLKDsbVQeKupqnglvGKYvrGKSWVdkVxcDshjbnD6wEOXya2t6MDQ777-_WxBDjLwlEOBkIuNJk2AwrDJ1_wyDDeiC_7y1Yxoe94UsDi7LiUjOF9QAyZ1OGMwVtP5/s1600/mvnc_fail.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiuCkxlVziRcFC-u_iLRLKDsbVQeKupqnglvGKYvrGKSWVdkVxcDshjbnD6wEOXya2t6MDQ777-_WxBDjLwlEOBkIuNJk2AwrDJ1_wyDDeiC_7y1Yxoe94UsDi7LiUjOF9QAyZ1OGMwVtP5/s320/mvnc_fail.png" width="224" /></a></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
</div>
David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com0tag:blogger.com,1999:blog-5837865295676240265.post-75842395527302166832012-11-01T17:56:00.000+11:002012-11-01T17:56:21.989+11:00Rant: Stupid Maven<div dir="ltr" style="text-align: left;" trbidi="on">
I'm back in the Java world again, and already I'm annoyed. The Ruby world isn't without its problems but Ruby problems are generally much easier to solve when they do appear. You generally don't end up wanting to punch the screen and scream "<i>give my life baaaack!</i>" (Thor's internals get me close though.)<br />
<br />
Today's problem: Maven.<br />
Now, I've been a big fan and advocate of Maven for years. Therefore, upon returning to the Java world I happily sought out my good friend Maven and started using it immediately with my new project. ...Only to have problem... after problem... after problem... So now I'm going to vent.<br />
<br />
<ul style="text-align: left;">
<li>TO DEPLOY to a local instance of Nexus you need to plonk big blob of shit (ie. XML) in every single project POM file. Adding <span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><distrubutionManagement></span> in user profile's settings.xml doesn't work. WTF! In Maven-land, any artifact can be deployed to any repository so long as the repo configuration allows it. A beauty of Maven is the interoperability of artifacts & repositories, this is, a project that can deploy to Maven Central can also deploy to an intranet Nexus or Artifactory; no code needs to change; the protocol is standard. Forcing the storage of repo settings alongside the project's build instructions doesn't seem very bright to me. It's nice... but how to build a project should depend on where a project's gonna go after its built (especially when there's no impact on build anyway). In reality they are separate concerns but the system doesn't allow that separation. If repo info could be stored in either project <i><b>or</b></i> user-profile settings so that people could decide based on the project circumstances then fine, but as it stands currently, it is infuriatingly not the case and not supported.</li>
</ul>
<ul style="text-align: left;">
<li>SO, NOW to make things work, I need to have the same parent pom for all of my projects that I plan to deploy to my local Nexus. The alternative would be to copy & paste the same 20 lines or so of XML into each project: umm no. What's annoying about this is <b>a)</b> it's something else to maintain (as opposed to settings.xml which wouldn't have to be deployed or available as an artifact), <b>b)</b> it introduces a new dependency (literally, not runtime) to all my projects now, great; and <b>c)</b> Maven poms use single-inheritence -- no mixins; to inherit from a different parent I have more stuffing around to do. Bad architecture. Annoying.</li>
</ul>
<ul style="text-align: left;">
<li>Parent POMs (multi-module or standalone) allow you to configure plugins, however those plugin settings aren't automatically inherited in child projects. For settings I think you need to redeclare the plugin in the child project (not 100%, I don't even care anymore). What really bugs me is that version specifications are not inherited. WTF GUYS! So if I have a multi-module project (and I tried this) where in I declare the version of a plugin in the parent pom, the child does not use the specified version of the project. OMG. I tried with <span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><inherited>true</inherited></span>, I tried in both <span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><plugins></span> and <span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><pluginManagement></span>, it doesn't matter. Now I guess I'm supposed to declare the versions in as properties and specify plugin versions in every single child module instead? Not only am I averse to duplication for the obvious reasons, but now my pom files are all massive! Now I googled this and apparently this is deliberate because "<i>you should always specify specific versions, inheriting versions is evil and disallowed"</i>, yada yada. Yep, well it's one thing in the context of isolated projects but in the context of multiple modules comprising a single project, it's the opposite. The modules are components that are meant to be built together to create a larger system. Declaring certain plugin versions at project-scope is entirely reasonable and has many advantages. Secondly, when you don't specify a plugin version, where does it come from? The build still works so it's coming from somewhere, right? It comes from the Maven super POM which is like a template that all poms inherit. In it, default plugin versions are specified left-right-and-centre, and they are inherited and used without explicit specification or "evil" consequences (although there are some unless the release plugin has been updated to hardcode all plugin vers in child poms during release - haven't used it in a few years). Double standards.</li>
</ul>
<ul style="text-align: left;">
<li>Finally XML. Come on. It's 2012. XML has always been a tedious, ugly, unacceptably verbose format. The reasons that made it appealing in 1998 are no longer valid in my opinion. Especially not in the context of pom.xml. One the best things about Maven is that you don't have to write a bunch of crap for every project in order to get it to build in an automated fashion; Maven allows you to simply say "<i>I'm blah v1.0 and I need v2 of X and v3.4 of Y to build. See ya!</i>" and it takes care of the rest so long as you follow its standards. Add a few dependencies though and add a single command (argLine) to surefire (the Maven testing plugin) and you're looking at over 100 lines of XML. Writing everything manually in Ant would only come to about 80 or so and it uses XML too! There was a project called Maven Polygot a while back that was supposed to allow the pom to work in different formats but it seems dead or at least progressing slowly enough that it is moot anyway. Progress is pretty generally slow in Maven-land. <u>Point:</u> In this day and age I resent having to use XML almost anywhere. Have you ever filled in a form by hand, a tax return, a car rego, and seen:
<blockquote><span style="font-family: Courier New, Courier, monospace;">Phone Number: ________________ /Phone Number</span>
</blockquote>
I haven't. I really dislike XML and I'm glad it's finally starting to trend away. Look at how concise things could be:</li>
<ul>
<li>pom.yml -- <a href="https://github.com/mrdon/maven-yamlpom-plugin/wiki">https://github.com/mrdon/maven-yamlpom-plugin/wiki</a></li>
<li>Gradle -- <a href="http://www.gradle.org/docs/current/userguide/artifact_dependencies_tutorial.html">http://www.gradle.org/docs/current/userguide/artifact_dependencies_tutorial.html</a></li>
<li>Buildr -- <a href="http://stackoverflow.com/questions/1015525/why-use-buildr-instead-of-ant-or-maven">http://stackoverflow.com/questions/1015525/why-use-buildr-instead-of-ant-or-maven</a> and scroll down a little.</li>
</ul>
</ul>
<div>
Rant.stop.</div>
</div>David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com0tag:blogger.com,1999:blog-5837865295676240265.post-35199949739240744022012-09-12T14:47:00.001+10:002012-09-12T14:48:14.728+10:00Problems with at_exit{}, exit(), and RSpec<div dir="ltr" style="text-align: left;" trbidi="on">
I had an interesting problem the other day, working on a Ruby project of mine. I ran my tests:
(<em>note: I have "<code>rak</code>" aliased to "<code>bundle exec rake</code>"</em>)
<blockquote><code>rak test</code></blockquote>
which internally expands to the equivelent of:
<blockquote><code>rak test:spec test:int</code></blockquote>
which runs my specs and integration tests in that order.
<p>
Then an odd thing happened. My specs failed but then the integration tests ran anyway and scrolled the spec failure off the screen to report happy success. After some digging around I discovered the following:
<ul>
<li>I'd had this test failure for a few commits without noticing.</li>
<li>My CI builds on Travis CI were all reporting success (although the failure message was there in the build logs should one manually check.)</li>
<li>Rake itself and RSpec's rake task were both fine.</li>
<li>Running my tests directly with the <code>rspec</code> CLI, I was getting an exit code of 0 on both success and failure.</li>
</ul>
</p>
<div class="section_title">The Problem</div>
<p>RSpec was returning an error/exit/status code of 0 despite test failure. It should be non-zero so that external processes like Travis CI and Rake can determine that something's gone wrong and react accordingly.</p>
<p>I'm going to cut a tedious story down to the result here. After investigation I learned that <code>rspec</code> worked as expected again when I avoided this piece of code in my tests:
<script src="https://gist.github.com/3704227.js?file=prob.rb"></script>
What this piece of code does is:
<ol><li>create a temporary directory the first time it's called</li>
<li>reuse that temporary directory on subsequent calls</li>
<li>remove the directory at the end of the process's lifecycle</li>
</ol>
</p>
<p>Why does that affect RSpec returning non-zero on failure? Because RSpec itself doesn't run immediately; it wants to wait until all of your specs have been loaded first. The way it accomplishes that is it registers itself to run via an <code>at_exit</code> block and then calls <code>exit</code> when it's finished with your specs. Still sounds like there's no problem right? Well this is what happens from that point on...
<ol>
<li>RSpec finishes running tests and calls <code>exit</code> with 0 for success or 1 for failure.</li>
<li>Accordingly, Ruby creates an instance of <code>SystemExit</code> and plonks it into the <code>$!</code> global variable.</li>
<li>Now that the process is shutting down, my <code>at_exit</code> in the snippet above starts running.</li>
<li>My cleanup code (correctly, validly and legally) runs Ruby's <code>FileUtils.remove_entry_secure</code> to remove the temporary directly created during tests. This isn't a problem in itself.</li>
<li>Here's the gotcha: <code>FileUtils.remove_entry_secure</code> removes the directory and sets <code>$!</code> to <code>nil</code> to indicate that no exception occurred.</li>
<li>Ruby ends the process and sets the exit code to the result of <code>$!.status</code> which was lost in the previous step.</li>
</ol></p>
<div class="section_title">The Solution</div>
<p>That was the problem. Now what's the solution?<br/>
Simple. Just take care to preserve the exit status in your <code>at_exit</code> so that it ends with what it started with. Here is a helper method that I created:
<script src="https://gist.github.com/3704227.js?file=soln.rb"></script>
(You can also view this file
<a href="https://github.com/japgolly/golly-utils/blob/master/lib/golly-utils/ruby_ext/kernel.rb">directly on Github in a utility library of mine</a>.)
</p>
<p>Then the solution becomes simply use <code>at_exit_preserving_exit_status</code> instead of <code>at_exit</code> in first snippet, and everything works again! Happy days!
<p>
</div>
<div class="section_title">TL;DR: Conclusion</div>
<p>
Be careful that you don't corrupt the value of <code>$!</code> when using <code>at_exit</code>. If you're not careful (or don't use a handy, safe function like presented above), then you can corrupt the exit status of RSpec in particular, and other libraries that work in a similar fashion.
</p>David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com5tag:blogger.com,1999:blog-5837865295676240265.post-38285780631625058332012-09-12T13:45:00.000+10:002012-09-12T13:45:01.553+10:00I Have Returned<div dir="ltr" style="text-align: left;" trbidi="on">
Over the past 4 months I've spent a lot of time overseas on holiday. I did a bunch of Asia with some mates, I visited India with my girlfriend. It was great fun and good to get away and have some new experiences, be in situations that you normally wouldn't (or even want to in some cases). I enjoyed myself and I'm glad I did it.<br />
<br />
And now, I want no more of it for at least 6 months! I've been on 13 planes over the last 4 months. I'm tired of travelling. Now that I'm back home for good, I'm looking forward to finally being able to focus on work again.<br />
<br />
Thus, I'll start giving this blog attention again. I have returned.<br />
<br /></div>
David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com0tag:blogger.com,1999:blog-5837865295676240265.post-31272976991019914122012-04-23T10:00:00.001+10:002012-04-23T10:16:28.711+10:00Ruby Mutex Reentrancy<p>
This morning I was making some Ruby code of mine thread-safe which is always fun. (I'm serious btw. I frikking love multithreaded programming!) In doing so I came across something that I found a bit surprising.
</p>
<p>
Consider the following snippet:
<script src="https://gist.github.com/2467503.js?file=reentrancy-mutex.rb"></script>
Think it will work?
Let's try...
<blockquote><code><pre>
<internal:prelude>:8:in `lock': deadlock; recursive locking (ThreadError)
from <internal:prelude>:8:in `synchronize'
from reentrancy.rb:5:in `block in <main>'
from <internal:prelude>:10:in `synchronize'
from reentrancy.rb:4:in `<main>'
</pre></code></blockquote>
<span style="font-size:14px; font-weight:bold; color:red">Shocking!</span>
<br/>Mutex is not reentrant. Wow. Ok. Let's try something else...
</p>
<p>
Let's change that <code>Mutux</code> into a <code>Monitor</code> and try again.
<script src="https://gist.github.com/2467503.js?file=reentrancy-monitor.rb"></script>
Alrighty, let's put on fresh underwear and give it a whirl...
<blockquote><code><pre>
Monitor is reentrant.
</pre></code></blockquote>
Ah, the world makes sense again. If I had to code my own reentrancy I would've cried and hated Ruby a little bit. My love and faith in Ruby remains, yay!
</p>
<div class="section_title">Is There A Cost?</div>
<p>
Nothing is free. Is there a performance penalty? Time for some benchmarks.
<br/><br/>Here is a little benchmarking script that acquires and releases both a mutex and monitor 1 million times each:
<script src="https://gist.github.com/2467503.js?file=reentrancy-benchmark.rb"></script>
Benchmarking results:
<blockquote><code><pre>
user system total real
Mutex 0.400000 0.000000 0.400000 ( 0.406259)
Monitor 0.870000 0.010000 0.880000 ( 0.864888)
</pre></code></blockquote>
Ouch, monitor takes over the double the time that mutex does. That's the trade-off.
</p>
<div class="section_title">What About JRuby</div>
<p>I'm curious, let's try JRuby too. We'll change <code>bm</code> to <code>bmbm</code> and fire it up.
<blockquote><code><pre>
Rehearsal ---------------------------------------------
Mutex 0.571000 0.000000 0.571000 ( 0.539000)
Monitor 2.012000 0.000000 2.012000 ( 2.012000)
------------------------------------ total: 2.583000sec
user system total real
Mutex 0.321000 0.000000 0.321000 ( 0.321000)
Monitor 1.696000 0.000000 1.696000 ( 1.696000)
</pre></code></blockquote>
Wow, Monitor is 5.3x slower when using JRuby!!! Hmmm, I suspect JIT just need more time to warmup. Here's a new benchmarking script with a big warmup:
<script src="https://gist.github.com/2467503.js?file=reentrancy-benchmark-jruby.rb"></script>
And the results:
<blockquote><code><pre>
> jruby --1.9 --fast reentrancy-benchmark-jruby.rb
Warmup #1/20
...
Warmup #20/20
user system total real
Mutex 0.357000 0.000000 0.357000 ( 0.357000)
Monitor 0.768000 0.000000 0.768000 ( 0.768000)
</pre></code></blockquote>
Ok, that's on-par with the MRI results. Mutex is fast off-the-bat with JRuby where as Monitor will be a lot slower at first then decrease to a little over double the speed of mutex.
</p>
<div class="section_title">Conclusion</div>
<p style="font-size:16px"><b>Mutex:</b> No reentrancy. Fast, less than half the speed of Monitor.
<br/><b>Monitor:</b> Reentrancy. Slow, little over twice as slow as Mutex.
</p>David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com3tag:blogger.com,1999:blog-5837865295676240265.post-70363698641013815552012-04-18T20:27:00.000+10:002012-04-23T11:22:46.677+10:00Ruby JSON LibrariesOver the last 5 years or so I'd been away from the world of Ruby. I still used Ruby at work and home for various little things [cos it's <i>awesome!</i>] but that's quite different to living in it, especially seeing its community is one of the most fast-paced I've seen. So when I came back recently and I needed a JSON library, I went searching and found (what felt like) 100 different JSON libraries...<br />
<br/>
Long story short, I benchmarked them. Feel free to skip to the end of this post to just get the conclusion and be on your way.
<div class="section_title">What Was Used</div>
<p>
<table class="pretty">
<tr><th>NAME</th><th>VERSION</th><th>BUILD</th></tr>
<tr><td>MRI Ruby</td><td>1.9.3p125</td><td><pre>ruby 1.9.3p125 (2012-02-16 revision 34643) [x86_64-linux]</pre></td></tr>
<tr><td>JRuby</td><td>1.6.7</td><td><pre>jruby 1.6.7 (ruby-1.9.2-p312) (2012-02-22 3e82bc8) (OpenJDK 64-Bit Server VM 1.7.0_03-icedtea) [linux-amd64-java]</pre></td></tr>
<tr><td style="text-align:center">"</td><td>1.7.0.dev</td><td><pre>jruby 1.7.0.dev (ruby-1.9.3-p139) (2012-04-15 b4b38d4) (Java HotSpot(TM) 64-Bit Server VM 1.7.0_03) [linux-amd64-java]</pre></td></tr>
<tr><td>OpenJDK</td><td>1.7.0_03</td><td><pre>OpenJDK Runtime Environment (IcedTea7 2.1) (ArchLinux build 7.b147_2.1-3-x86_64)
OpenJDK 64-Bit Server VM (build 22.0-b10, mixed mode)</pre></td></tr>
<tr><td>Oracle Java</td><td>1.7.0_03</td><td><pre>java version "1.7.0_03"
Java(TM) SE Runtime Environment (build 1.7.0_03-b04)
Java HotSpot(TM) 64-Bit Server VM (build 22.1-b02, mixed mode)</pre></td></tr>
<tr><td>MultiJson</td><td>1.3.2</td><td><pre>c73bc389fa1b0b1c0b8225ea77ff3e2dee312304</pre></td></tr>
</table>
</p>
<p>
The following JSON libraries were tested:
<table class="pretty">
<tr><th>NAME</th><th>GEM NAME</th><th>VERSION</th></tr>
<tr><td>Optimized JSON (Oj)</td><td><pre>oj</pre></td><td>1.2.4</td></tr>
<tr><td>YAJL</td><td><pre>yajl-ruby</pre></td><td>1.1.0</td></tr>
<tr><td>JSON JRuby</td><td><pre>json-jruby</pre></td><td>1.5.0</td></tr>
<tr><td>JSON Pure</td><td><pre>json_pure</pre></td><td>1.6.6</td></tr>
<tr><td>JSON gem</td><td><pre>json</pre></td><td>1.6.6</td></tr>
<tr><td>OkJson</td><td><pre>okjson</pre></td><td>Version packaged with MultiJson 1.3.2</td></tr>
</table>
</p>
<p>
I created a little app to benchmark each library that performs two functions 100,000 times and records the time taken. Said two functions are:
<ol>
<li><b>[Writing]</b> Generates JSON for Ruby data structure:<pre>{
a: 2,
b: (1..50).to_a, # i.e. an array of 1,2,3,4,5,6, ... ,49,50
c: %w[asf xcvb sdfg sdf gfsd],
d: {
omg: 'hedfasgdsfg',
wewr: 34,
sfgjbsdf: %w[sdfg sdfgsdfgnj klj kj hkuih ui hu kjb bkj b sdfg],
},
}</pre></li>
<li><b>[Reading]</b> Parses this JSON:<pre>
{"a":2, "b":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50], "c":["asf","xcvb","sdfg","sdf","gfsd"], "d":{"omg":"hedfasgdsfg", "wewr":34, "sfgjbsdf":["sdfg","sdfgsdfgnj","klj","kj","hkuih","ui","hu","kjb","bkj","b","sdfg"]}}</pre></li>
</ol>
</p>
<p>
If you're interested you can grab the code here: <a href="https://github.com/japgolly/WebServerBenchmark/tree/json_libraries">https://github.com/japgolly/WebServerBenchmark/tree/json_libraries</a><br/>
You're free to read it, play with it, hack it, print it and eat it, make love to it; It's all good.
</p>
<p>Finally, these tests were performed on a Q9550 with 8GB RAM running Arch Linux 64-bit.
<pre>➤ uname -a
Linux golly-desktop 3.3.2-1-ARCH #1 SMP PREEMPT Sat Apr 14 09:48:37 CEST 2012 x86_64 Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz GenuineIntel GNU/Linux</pre>
</p>
<div class="section_title">Results: First Cut</div>
<p>
The numeric axis represents number of seconds taken to perform 100,000 operations.
<br/>Less is better.
<table>
<tr><th></th><th>Linear Graph</th><th>Logarithmic Graph</th></tr>
<tr><th>MRI</th><td><img src="http://img688.imageshack.us/img688/1571/mrilinear.png"/></td><td><img src="http://img256.imageshack.us/img256/8228/mrilog.png"/></td></tr>
<tr><th>JRuby (on Oracle Java)</th><td><img src="http://img688.imageshack.us/img688/2128/jrubylinear.png"/></td><td><img src="http://img854.imageshack.us/img854/7482/jrubylog.png"/></td></tr>
</table>
Wow. Two things are immediately obvious:
<ol>
<li>OkJson is extremely slow on both Ruby implementations. Whatever its design goals, performance isn't one of them.</li>
<li>JRuby runs YAJL like my grandmother runs marathons.</li>
</ol></p>
<p>
I really thought JRuby would perform better. Maybe 1.7 will be better. I know the JRuby team expect significant performance gains; I wonder if they've implemented them yet... Let's try using the latest dev version of JRuby 1.7.0!
</p>
<p>Also I think I'll try using JRuby 1.6.7 with the OpenJDK implementation of Java and see if that gives better results.</p>
<div class="section_title">Results: Lots of JRuby Love</div>
<p style="color:#d00; font-weight:bold">[Update 2012-04-23: New JRuby library discovered, see conclusion.]</p>
<p>
Alrighty, done. Here are the results: (<b>note:</b> using a logarithmic scale here again)
<img src="http://img577.imageshack.us/img577/777/allbar.png" />
And the same thing expressed differently:
<img src="http://img801.imageshack.us/img801/7089/allline.png" />
</p>
<p>Jeez, it's not getting much better for JRuby...</p>
<p>Ok, enough of this. Let's get rid of the council-working options <i>[hey, Aussies get that!]</i> and just look at the feasible ones.</p>
<div class="section_title">Results: The Finalists</div>
<p>
The numeric axis represents number of seconds taken to perform 100,000 operations.
<br/>Less is better.
<img src="http://img833.imageshack.us/img833/9600/significantbar.png" />
<img src="http://img440.imageshack.us/img440/7097/significantline.png" />
</p>
<div class="section_title">Conclusion</div>
<p style="font-size:16px">
If you use <span class="highlight1">MRI</span>, use <span class="highlight1"><code>Oj</code></span> unless you generate more JSON than you parse, in which case use <span class="highlight1"><code>YAJL</code></span>.
<br/><br/>
If you use <span class="highlight2">JRuby</span>, you've only really got one choice: <span class="highlight2"><code>json-jruby</code></span>.
You can expect roughly the same performance with OpenJDK, Oracle Java and the latest dev build of JRuby.
<br/><br/>
<span style="color:#d00; font-weight:bold">Update 2012-04-23:</span> <a class="highlight2" href="https://github.com/guyboertje/jrjackson">jrjackson</a> for <span class="highlight2">JRuby</span> is approx <b>4x faster</b> than <code>json-jruby</code> and faster than MRI. If you use JRuby, you want this!!
</p>
<br/><br/>David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com1tag:blogger.com,1999:blog-5837865295676240265.post-75191831914059478472012-04-02T16:22:00.000+10:002012-04-02T16:30:27.223+10:00I Have Blog<p>
And I decided I would create a new blog, the first in nearly 6 years.
</p>
<p>
And I decided I would overcome the ennui of asynchronous communication with a strategy (!), seeing things with new perspectives won through experience now that I'm the ripe, hoary age of 31.
</p>
<p>
And I decided I would bless this new blog with an irrelevant excerpt I love from a brilliant saga called <a href="http://en.wikipedia.org/wiki/Malazan_Book_of_the_Fallen">Malazan Book of the Fallen</a> by Steven Erikson.
</p>
<p><blockquote><i>
Her finger provided the drama, ploughing a traumatic furrow across the well-worn path. The ants scurried in confusion, and Samar Dev watched them scrabbling fierce with the insult, the soldiers with their heads lifted and mandibles opened wide as if they would challenge the gods.
</i></blockquote><p>
<p>
I don't know why I love that so much, but I do.
Hey! I <i>did</i> say it was irrelevant.
</p>David Barrihttp://www.blogger.com/profile/05426582122438954031noreply@blogger.com0