[{"content":"","date":"17 March 2024","externalUrl":null,"permalink":"/posts/","section":"Blog","summary":"","title":"Blog","type":"posts"},{"content":"","date":"17 March 2024","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"","date":"17 March 2024","externalUrl":null,"permalink":"/tags/goals/","section":"Tags","summary":"","title":"Goals","type":"tags"},{"content":"","date":"17 March 2024","externalUrl":null,"permalink":"/categories/leadership/","section":"Categories","summary":"","title":"Leadership","type":"categories"},{"content":" Introduction1 # New year’s resolutions and goals lists have traditionally been the way we declare our personal goals and aspirations for the year(s) ahead. Mostly this is done in a motivational and ceremonial fashion with no commitment to execution or seeing these goals through. More often than not our approach to executing these personal goals is reactive in nature, such that we only make a strong commitment to execute these goals out of necessity, for example being advised by the doctor to exercise in order to reduce high blood pressure.\nOn the contrary, we follow a more structured approach to our work goals and career paths at work as we rely on our employer to chart and structure this process. In this article I’m going to present a structured, proactive and visual approach to personal planning as an alternative to the humble goals lists and new year’s resolutions. This approach has worked well for me over the past few years, it has helped me to stay focused on achieving the goals that supported my personal vision. This approach is comprised of the following three phases:\nFigure 1. Mapping your journey\nPlan, you start of with a personal vision statement. You then brainstorm all of the goals that can help you achieve this vision and add them to a mind map under the relevant categories, henceforth I refer to this mind map as the “Vision Map” (see Figure 2) Select, you then look at each of these goals and try to fine-tune them to make them SMART goals, whatever you can’t make into a SMART goal just park it for later under an “Aspirations” group. Identify which goals are recurrent, e.g. habits and which ones are one-off goals. Then choose the goals you want to target for the month, quarter, year, etc. taking in to consideration your other commitments. Execute, you start the execution of your goals by using the right tools, e.g. reminders, planning calendars, etc. Use mental contrasting to help you visualize and overcome any obstacles that stand between you and achieving your goals. At the end ensure to reward yourself for goal achievement. Figure 2. A Vision Map of goal categories\nThe benefits of a structured, proactive and visual approach # Before I dive into these three steps let me first explain the benefits of adopting a structured, proactive and visual approach to personal goals planning.\nStructured # Starting with the first keyword “structured”, adding structure to our goals will help us clearly see them as a journey based on our personal vision, rather than a disconnected list of goals. This vision could be to lead a healthy happy life, to be a devoted parent and partner, to have a successful career, to mention just a few. It’s hard seeing this journey and measuring progress based on a collection of different goal lists that we have created over the years. We can always borrow structure and get inspirations from the various career path planning techniques that we go through with our employers throughout our work experience.\nProactive # “Success in the knowledge economy comes to those who know themselves—their strengths, their values, and how they best perform.”\nPeter F. Drucker\nThe second keyword is “proactive”, as I mentioned previously sometimes our goals are a reaction to a particular situation, for example having to complete a professional certification as a result of loosing a job. Taking more of an explicit and proactive approach helps us to plot our journey in advance and be in a better position to pivot or adjust depending on the current situation.\nPeter F. Drucker a renowned management consultant, writer and teacher states in his article Managing Oneself: “Success in the knowledge economy comes to those who know themselves—their strengths, their values, and how they best perform”. Adopting a proactive approach to our personal development and goals planning helps us to better achieve our vision, as Drucker puts it in his book Managing Oneself: “…You must be your own chief executive officer. That means it\u0026rsquo;s up to you to carve out your place in the world and know when to change course”.\nVisual # It’s always said that a picture is worth a thousand words, that is because visuals are more engaging and memorable than text alone. By incorporating images, diagrams, and sketches, people can better retain and recall information, leading to effective communication and better decision-making.\nDan Roam in his book The Back of the Napkin writes: “There is no more powerful way to prove that we know something well than to draw a simple picture of it. And there is no more powerful way to see hidden solutions than to pick up a pen and draw out the pieces of our problem.” Representing our goals visually helps us to better comprehend them and their connections to our bigger journey, it helps us to group and nest these goals, to add images, diagrams and sketches leading to more effective planning and decision making.\nStable Diffusion XL 1.0 (1024): a mindmap of goal icons floating in the sky\nMy use of visuals for planning goals started back in 2015, the busiest I’ve ever been in my life. In addition to my daytime job and fulfilling other family commitments, I was working on my PhD and developer community activities in the evenings (meetups, etc.). Back then, I felt a great deal was happening in my life that I needed a better way to map all what I was doing and enjoying to a journey of goals that add up to a vision. So I started drawing a simple mind map on the whiteboard to group and nest my goals.\nWhy a mind map?, I hear you ask. A mind map is a visual representation of information that is organized around a central concept, idea, or theme. It is a diagram that visually depicts the relationships between various concepts, allowing for a more structured and interconnected understanding of a topic. Mind maps are often used for brainstorming, organizing thoughts, problem-solving, and visualizing complex information. I found this mind mapping so effective that I used mind maps to flush out everything I needed to complete for my PhD.\nNow I hope you appreciate the value a structured, proactive and visual approach brings to personal goals planning. In the next sections I will explain the three phases of this approach, plan, select and execute.\nPlan # Our goals are usually defined based on a higher purpose, a vision that we set to ourselves, this could be to lead a healthy happy life, to be a devoted parent and partner, to have a successful career, to mention just a few. The planning phase is comprised of three steps:\nStep 1: Prepare a personal vision statement # The first step in the planning phase is to start with your personal vision statement, here is an elaborate example:\n“I envision myself as a devoted husband and father, nurturing loving relationships built on trust, respect, and shared experiences. Within my community, I aspire to be a pillar of support, fostering unity, and contributing to positive social change through acts of kindness and service. In my career, I am committed to continuous growth and innovation, leveraging my skills and experiences to make meaningful contributions, inspire others, and drive positive impact in the world.”\nStep 2: Brainstorm all goals # The next step is to think of goals that can help you fulfill this vision statement. Goals can be grouped into categories as shown in the vision map in Figure 3 below, e.g. Relationships, Finance, Lifestyle, etc. At this stage just brainstorm all possible goals you can think of and add them to the vision map. I use SimpleMind because I can access my mind maps on all my Apple devices, but you can use any mind mapping software that you prefer. With SimpleMind you can add notes, icons or images to goals (referred to as topics in SimpleMind) in addition to the ability to cross link them. Goals can be recurring such as regular habits e.g. exercising, healthy diet, etc. or one-off goals such as achieving a particular qualification or certification. Large goals can be decomposed into smaller sub-goals to make them attainable.\nFigure 3. An example Vision Map with some Career and Lifestyle goals\nStep 3: Make personal goals SMART # Now that you have created some goals, the final step in the planning phase is to finalize the goals by making them SMART e.g. Specific, Measurable, Achievable, Relevant and Time-bound. Consider the following for your SMART personal goals:\nSpecific: Clearly define what you want to achieve. Be specific about the outcome and the actions required. Example: Instead of a vague goal like \u0026ldquo;Exercise more,\u0026rdquo; make it specific, such as \u0026ldquo;Run for 30 minutes three times a week.\u0026rdquo; Measurable: Set criteria to track your progress and measure success. This involves quantifying your goal so that you can assess your achievements. Example: Instead of \u0026ldquo;Read more books,\u0026rdquo; specify the target, such as \u0026ldquo;Read one book per month.\u0026rdquo; Achievable: Ensure that your goal is realistic and attainable. Consider your resources, constraints, and the feasibility of reaching the goal within the given parameters. Example: If you are new to running, an achievable goal might be to \u0026ldquo;Complete a Couch to 5K race in six months.\u0026rdquo; Relevant: Align your personal goal with your vision statement, values, interests, and broader life objectives. Ensure that the goal is meaningful and contributes to your overall well-being. Example: If your broader life goal is to prioritize health, a relevant goal might be \u0026ldquo;Cook and eat at least three homemade, healthy meals per week.\u0026rdquo; Time-bound: Set a deadline for achieving your goal. This adds a sense of urgency and helps you stay focused. Example: Instead of \u0026ldquo;Learn a new language,\u0026rdquo; make it time-bound, such as \u0026ldquo;Achieve basic conversational proficiency in Spanish within six months.\u0026rdquo; For any goal that feels far fetched, blue-sky or can’t be made SMART at the present moment then move to it to the “Aspirations” category, this will act as a parking lot, giving you the ability to review and refine these goals at a later date and move them back into their categories when you feel the time is right. Below is an example SMART goal for achieving ITIL Practitioner certification in the next 3 months, with a commitment of 10 hours of study per week.\nFigure 4. SMART goal for achieving ITIL Practitioner certification\nSelect # After planning your goals and making them SMART, you move on to the Select phase. During this phase, you select goals to focus on for a predefined period, such as a month, a quarter, or a year. At this stage, it\u0026rsquo;s best to adopt a balanced and flexible approach to your goals, considering other aspects of your life such as relationships, well-being, and overall happiness. Strive for a balance that contributes positively to your overall quality of life, bearing in mind that life circumstances can change, and it\u0026rsquo;s okay to adjust your goals accordingly. It\u0026rsquo;s important to understand that pursuing goals requires time commitment, often leading to tradeoffs and sacrifices due to our limited hours in the day. For example, when I pursued professional certifications, I often had to forgo watching TV to make time for studying.\nTo highlight the goals I want to focus on for the year ahead, in SimpleMind I usually mark these goals with an icon e.g. a star and add a due date as well. (see Figure 3 \u0026amp; 4 above)\nExecute # Planning and listing our goals is the easiest part, in contrast the hardest part is finding the motivation, time and energy to commit to executing these goals. Some of the obstacles that hinder us from achieving our goals include internal factors such as self-doubt, lack of motivation, procrastination, etc. or external factors such as time constraints, or competing priorities. One approach that can help to address this is mental contrasting.\nMental contrasting is a cognitive strategy used to enhance goal pursuit and motivation. It involves contrasting the desired future outcome (the goal) with the current reality (obstacles, challenges, or potential barriers) in a systematic way. Using this technique individuals can gain clarity, motivation, and strategic insights to overcome obstacles and achieve their goals. This technique was developed by psychologist Gabriele Oettingen. Here\u0026rsquo;s how it works:\nIdentify the Goal: Start by clearly defining the specific goal you want to achieve. This could be anything from personal aspirations to professional objectives. Visualize the Outcome: Imagine the desired future outcome associated with achieving the goal. Picture yourself successfully accomplishing the goal and the positive consequences that result from it. Engage your senses and emotions to make the visualization vivid and compelling. Identify Obstacles: Next, identify potential obstacles, challenges, or barriers that stand in the way of achieving the goal. As mentioned before, these could be internal factors (such as self-doubt, procrastination, or lack of resources) or external factors (such as time constraints or competing priorities). Contrast Future with Reality: Contrast the desired future outcome with the current reality by mentally comparing the positive aspects of achieving the goal with the obstacles and challenges that may arise. This contrast highlights the gap between where you want to be and where you currently are. Develop Implementation Intentions: Finally, develop specific plans or implementation intentions to overcome the identified obstacles and increase the likelihood of goal attainment. These plans outline the specific actions you will take to address each obstacle as it arises. Whilst mental contrasting can help you gain clarity, motivation, and strategic insights to overcome obstacles, it’s crucial to start executing your goals, to think big, but to start small. As you achieve your goals, no matter how small, you will discover what motivates and demotivates you, which will help you craft a tailored formula for goals execution. I personally use a yearly calendar which I annotate with important dates, I also use posted notes for reminders, etc. Life circumstances can change, so regularly reflect on your goals, assess your progress, and be willing to adapt them as needed. Personal growth may lead to evolving aspirations, so feel free to adjust your vision map and goals priorities as required (see Figure 5 below).\nFigure 5. Adjusting your goals\nTo keep yourself motivated ensure to recognize and celebrate your achievements along the way, such recognition can greatly boast motivation. I motivate myself with little things like displaying certificates, medals (Figure 6 below), trophies and sharing achievements with friends, family and on Social Media, etc. Some might see sharing achievements as shameless self promotion, however in my opinion, sharing achievements can serve as a form of self-recognition and celebration of one\u0026rsquo;s hard work and accomplishments. It can be a way to acknowledge personal growth, build confidence, and reinforce motivation to continue striving for success. When done in a humble and genuine manner, sharing achievements can inspire others and foster a positive, supportive community as well.\nFigure 6. Getting motivated by displaying my running medals\nSummary # In this article we explored a technique for personal goals planning, a technique that is structured, proactive and visual. This technique is centered around a vision map, a customized mind-map used to brainstorm and group all goals that help us to achieve our personal vision. The technique has three phases: plan, select and execute. Since I started using this technique a few years back, it helped me to organize my personal goals and view them as a multi-year journey of personal growth rather than bullet lists of disconnected goals. Below is an example of my current vision map which includes previous goals both completed and abandoned, viewing these previous goals helps me learn about myself, my passions and evolving priorities.\n","date":"17 March 2024","externalUrl":null,"permalink":"/2024/03/17/mapping-your-journey-a-proactive-visual-approach-to-personal-goals-planning/","section":"Blog","summary":"“…You must be your own chief executive officer. That means it’s up to you to carve out your place in the world and know when to change course”","title":"Mapping Your Journey: A Proactive Visual Approach to Personal Goals Planning","type":"posts"},{"content":"","date":"17 March 2024","externalUrl":null,"permalink":"/tags/motivation/","section":"Tags","summary":"","title":"Motivation","type":"tags"},{"content":"","date":"17 March 2024","externalUrl":null,"permalink":"/tags/planning/","section":"Tags","summary":"","title":"Planning","type":"tags"},{"content":"","date":"17 March 2024","externalUrl":null,"permalink":"/categories/productivity/","section":"Categories","summary":"","title":"Productivity","type":"categories"},{"content":"","date":"17 March 2024","externalUrl":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":"It\u0026rsquo;s not just a job, it\u0026rsquo;s a hobby and a way of life. Opinions expressed are my own and not those of my employer.\n","date":"17 March 2024","externalUrl":null,"permalink":"/","section":"Technology with Passion","summary":"","title":"Technology with Passion","type":"page"},{"content":" Introduction and TLDR; # For me reading and writing are synonymous with learning and sharing. In this short post I reflect on some of the professional development reading and learning that I’ve done in my spare time during 2023 and share with you some of the things that worked for me and helped me achieve my learning goals. If you are like me struggle to sit down to read books or watch videos, etc. then read on.\nThey key takeaways from this post are:\nIdentifying the optimal recipe to consistently tackle a learning task is as important as embarking on the task itself. Small, regular habits prove to be more enduring over time compared to one-off large efforts. Engage with individuals who share similar interests, as they may offer valuable insights that contribute to your quest for discovering your own effective learning recipe. An Effective Reading Recipe # I have some of the classic tech books (e.g. The Lean Startup) in my bookshelves for almost a decade now, but in all honestly I never got the time to actually sit down and read some of these books cover to cover. With the exception of programming and Geoffrey Moore’s books (Crossing the Chasm, The Chasm Companion \u0026amp; Inside the Tornado), I struggled to finish some of the others. I just could not find the time to sit down and read because I’m always moving, be it exercising, doing chores around the house or driving. I was chatting to a colleague last year about this and she suggested to try Audible, which I did and voila! that was a turning point for me. In my quest to find my own recipe I like to talk to like-minded people and during these chit-chats I always find some useful tips or inspirations.\nI started listening to some fiction books for which I already owned a printed copy, the experience was enjoyable and convenient. I then started to listen to some of the professional books that were decorating my bookshelves. Now I can listen whilst driving or exercising, and because I owned some of the printed books, I could mix between book reading and listening, specially when I have to sit down anyway, for example during traveling or waiting for appointments, etc.\nCapturing notes was also an important activity for me specially with interesting topics or topics that I want to recall later. In the case of a printed books I just highlight the section of interest and in case of Audible books, Audible provides the ability to bookmark a location with a note. So this was my recipe for reading, basically a mix of Read+Listen, it worked very well for me during 2023. Here is a list of the professional development books I’ve read / listened to during 2023:\nThe Lean Startup, by Eric Ries Radical Candor, by Kim Scott Powerful: Building a Culture of Freedom and Responsibility, by Patty McCord How to Write Non-Fiction: Turn Your Knowledge into Words, by Joanna Penn Accelerate: Building and Scaling High Performing Technology Organizations, by Nicole Forsgren, et al Think Like Amazon, by John Rossman I’m looking forward to continue with these books during 2024:\nNo Rules Rules: Netflix and the Culture of Reinvention, by Reed Hastings and Erin Meyer The Phoenix Project, by Gene Kim, Kevin Behr and George Spafford (already halfway through the book) The Unicorn Project, by Gene Kim Cloud Strategy: A Decision-Based Approach to Successful Cloud Migration, by Gregor Hohpe The Pyramid Principle: Long in Writing and Thinking, by Barbara Minto The Culture Code, by Daniel Coyle The Five Dysfunctions of a Team, by Patrick Lencioni The Quest for Professional Certifications # I always loved certifications, you learn and you get rewarded for this learning with a certificate. Based on my experience with some of the AWS certifications during 2022 I found out the following about my learning habits:\nI’m deadlines driven, having a cutoff date gives me a clear timeline to learn and imposes some desirable pressure as well, otherwise without a deadline or pressure other commitments will take over. I prefer a condensed learning style, ideally over 4 to 6 weeks, but not more than this to ensure the information is fresh on my mind. Otherwise longer periods of studying mean I’ve to repeat the content again to refresh, so double the effort. I prefer reading over watching videos, so I get the training course slides and go over these. I only watch the videos for topics I’m not familiar with. As I mentioned above finding time to sit to watch videos or read slides might be a challenge, in this case, depending on the topic if I won’t miss much by not looking at the slides then I simply listen to the video whilst doing physical activity. My recipe for certifications is to lookup my calendar and choose a date for the exam, anywhere from 1 week ahead to 6 weeks and then just book the exam, the rest will sort itself out.\nHere are the AWS certifications I’ve managed to clear during 2023:\nYo Aprendo Español Con Duolingo # Yes I started learning Spanish during 2023, I don’t know how this came by, but I’ve embraced it. Although not directly a professional development activity, there are many micro habits to acquire from learning a new language which could prove useful in professional settings. My story with wanting to learn Spanish started back in Jan 2021 just before the COVID pandemic when I travelled to Mexico on a work trip. I travelled to a few states to present to multiple audiences, and in some instances we needed to have a translator, during those moments I wished I could speak Spanish so I could natively convey my passion of the topic I was speaking about. It wasn’t until last summer when we visited southern Spain as part of a family holiday that I started learning Spanish. I got inspired by my daughter who is learning Italian on Duolingo reaching a 500 day streak by making it a habit to do a little everyday. The recipe here is “little but consistent and every little helps”\nMaking 2024 A Year of Writing and Sharing # My last personal blog post was in 2016, the year during which I had to write my PhD thesis. I won’t go into details on how difficult and tasking this was, having to juggle this with work and family commitments, mental blocks, etc. (if you are interested in my part-time PhD experience, checkout my blog post reflecting on the whole experience “How I Satisfied My Passion for Software Development and Open-Source by Doing a Part-Time PhD”.\nTo be honest after completing my PhD thesis I felt I’m done with writing outside of work, my writing batteries were depleted for good. However, almost 8 years on as I look into 2024 I feel those batteries are still rechargeable, I feel a flare for writing again starting here and now!\n","date":"7 January 2024","externalUrl":null,"permalink":"/2024/01/07/a-year-of-reading-and-learning/","section":"Blog","summary":"For me reading and writing are synonymous with learning and sharing. In this short post I reflect on some of the professional development reading and learning that I’ve done in my spare time during 2023 and share with…","title":"A Year of Reading and Learning","type":"posts"},{"content":"","date":"7 January 2024","externalUrl":null,"permalink":"/tags/audible/","section":"Tags","summary":"","title":"Audible","type":"tags"},{"content":"","date":"7 January 2024","externalUrl":null,"permalink":"/tags/productivity/","section":"Tags","summary":"","title":"Productivity","type":"tags"},{"content":"","date":"7 January 2024","externalUrl":null,"permalink":"/tags/reading/","section":"Tags","summary":"","title":"Reading","type":"tags"},{"content":"Resources Management Database for Royal Mail Coventry Delivery Office\nMy graduation project is based on module TM422. I\u0026rsquo;ve successfully gathered the requirements, designed and implemented a database system to manage the HR and jobs allocation in Royal Mail\u0026rsquo;s delivery offices. The database was implemented using SQL and included relational constraints, views and stored procedures\nStarting from requirements I\u0026rsquo;ve worked on the analysis of the delivery office requirements and converting these requirements into a database by going through the relational database theory:\nEstablish requirements (Data requirements) \u0026ndash;\u0026gt; Data Analysis (Conceptual data model) \u0026ndash;\u0026gt; Database design (Logical schema) \u0026ndash;\u0026gt; Implementation (Schema and database)\nResources Management Database for Royal Mail Coventry Delivery OfficeDownload\n","date":"10 December 2020","externalUrl":null,"permalink":"/bsc-graduation-project/","section":"Technology with Passion","summary":"Resources Management Database for Royal Mail Coventry Delivery Office","title":"BSc Graduation Project","type":"page"},{"content":" Photo kindly captured by Adel\nA version of this article was translated to Japanese.\nDate: 15th July 2016\nRetrospective - Back to the Future # At the end of the graduation day, the Vice-Chancellor of the university delivered a powerful speech. Words of wisdom that are going to stay with me for the rest of my life. \u0026ldquo;Be a source of energy, be positive, motive and inspire others\u0026hellip;\u0026rdquo;.\nDate: 15th May 2016\nA slight detour - What is this? # I love it when you call a fellow developer over to show them some code or a stacktrace and they say in exclamation \u0026lsquo;What is this?\u0026rsquo;. Sometimes not so nice words come before or after the question, which I will refrain from quoting in my blog . Now, what is this, why did I write it and why are you investing a few minutes of your life reading it? Well, like many passionate developers and engineers who hack stuff in their spare time, I\u0026rsquo;m passionate about software development and new technologies. Luckily for us:\nProgramming seems to be a relatively unique field because for many people it is both their job and their hobby, so they enjoy programming in their spare time. [Sara Miles on StackExchange]\nIf you didn\u0026rsquo;t think this introduction is funny, then you are probably not a software developer. Back to the point, to satisfy my passion for software development and new technologies, to become a better developer, researcher and technical writer, by 2010 I embarked on a part-time, self-funded PhD at the University of Reading whilst working full-time. Why part-time? because I had to work, leaving work for 3 years to do a full-time PhD wasn\u0026rsquo;t an option.\nIn this short article I share with you some of the experiences, challenges and achievements of this PhD journey. I\u0026rsquo;m hoping this article will somehow help, motivate or inspire you or someone out there to do a part-time MSc or PhD besides all your/their other commitments. It\u0026rsquo;s not easy and the journey is not without difficulties, but if you have the passion about what you are going to research, then it\u0026rsquo;s doable and fun. Note, my experience is based on the PhD process in the United Kingdom, you might have a different process in your country which makes it difficult to complete a PhD part-time. This page is my diary to reflect back on many things and will be updated regularly now that I\u0026rsquo;ve spare time, so come back for updates.\nWarning # Although this article seems structured and straightforward, the PhD process isn\u0026rsquo;t, it\u0026rsquo;s a journey full of ups and downs. So by no mean this article serves to hint that doing a PhD part-time or full-time for that matter is easy. It\u0026rsquo;s not, for part-time in particular you need a great deal of energy, determination, perseverance, persistence, patience and time management to get it done besides your other commitments. You need to give up a few things in your personal life and it\u0026rsquo;s not for everyone, just like starting \u0026lsquo;your own\u0026rsquo; business is not for everyone. The journey was hard, I nearly dropped off after 2 years, but I persisted and soldiered on. I\u0026rsquo;ve had a few health problems from sitting extended hours on the computer, sometimes 14+. I had to change supervisor once as well, which was a stressful experience. I gave up watching TV and reduced my social life to an absolute minimum, I had a great supportive wife who has a done a full-time PhD and understood what I was going through. I hardly had a free weekend. Additionally, being an independent IT consultant helped, such that I can decide to take on short contracts or take time off between contracts to focus on the research. Not buying that X5 and saving the money for the rainy day also helped with being able to take time off work when needed. Furthermore, pressure gets the best out of me, I\u0026rsquo;m used to studying and working at the same time from the undergraduate level, so such experience helped greatly in this journey. I\u0026rsquo;m not stating a rule or generalising that everyone can do it, but rather sharing my experience, so proceed with caution, your mileage may vary, and bear in mind that everyone is different. I believe in God and I believe it\u0026rsquo;s only with the grace of God Almighty that I was able to endure all of this and complete the journey.\nWhy do a part-time PhD? # I was working full-time, had a family with two children (my 3rd child arrived after I started), with all this why on earth did I want to do a PhD?\nFulfilling a life long dream # First, to fulfil a life long dream. Since I was young my life ambition was to do a PhD to follow the footsteps of my dad. I was born half-way through his PhD study and when I grew up he used to tell us about his experiments and how hard it was to write a thesis back then. Imagine in the mid seventies, with a typewriter, citation was a nightmare, Tipp-Ex and lots of papers everywhere. He once forgot his draft thesis in the bus and that was a disaster, luckily it was returned safely to the university. Makes one appreciate the luxuries we enjoy in this digital age!\nBecoming a better engineer and researcher # Second, based on my experience doing a part-time MSc — which was done during similar conditions — I found that I learned a great deal, became a better software developer and developed a fully functioning SaaS offering. I was able to explore new technologies during my MSc dissertation which benefited me greatly at work and I was able to share my knowledge with others. I\u0026rsquo;ve managed to align the MSc study with work and career and as a result of the technologies I\u0026rsquo;ve learnt and the experience I\u0026rsquo;ve gained I was promoted to a principal consultant at work!. As someone who is passionate about technology, software development and Open Source, doing a PhD is another way to contribute to the wider community by following a scientific methodology to advance the state of the art. I thought to continue this journey of exploration and learning by doing a PhD. All what I needed was self discipline and good time management skills, I also had a good supervisor that guided me through the process. I enjoyed being under pressure and was able to manage my time efficiently between work, family and research commitments. I already feel I\u0026rsquo;ve become a better technical person, I\u0026rsquo;m already feeling the benefits of the PhD journey, which I can summarise as follows:\nProud of my work and making original contributions. I can tackle complex technical problems in a methodological and scientific manner. I can effectively research complex technical topics and suggest novel solutions. My technical writing has improved considerably, in particular after writing a thesis with 90K words. I can constructively evaluate and criticise the work of others. I can easily articulate and explain my approach to others. I accept criticism from others regarding my own work and use such criticism to improve it. I\u0026rsquo;ve learnt many technologies and became more acquainted with and passionate about the Open Source ecosystem. It was a win win situation, I was reading about new technologies, reviewing journal papers, challenging myself to understand what others have done and find gaps in their work. In addition to reading publications and journal papers, writing code and developing prototypes were also great fun that consumed numerous weekends. The hardest and painful part was writing, lots of writing, but it had to be done, as one professor once said to me \u0026ldquo;The PhD is reading and writing\u0026rdquo;.\nThe journey of a lifetime # Here is a brief timeline showing key milestones during my PhD journey. I was studying in a part-time working-away mode, which meant the university didn\u0026rsquo;t have to supply me with a desk, computer, etc\u0026hellip; Additionally, this meant I paid the cheapest tuition fees, plus I didn\u0026rsquo;t need any physical hardware for my work because the computer lab for my experimentation work was all up in the cloud somewhere in Google\u0026rsquo;s datacentres (Google Cloud Platform). This doesn\u0026rsquo;t mean that I was working away in a rabbit hole and didn\u0026rsquo;t see the university. In fact I live 10 minutes (walk) away from the university and I used to see my supervisor almost weekly, sometimes in-person and other times over Skype. I attended many postgraduate training courses (literature review, thesis writing, etc\u0026hellip;). I attended other conferences, seminars, etc\u0026hellip; at the university.\nDate Milestone Artifact 17/10/2010 PhD proposal accepted 07/09/2012 07/09/2013 Suspended my study for a year. Wasn\u0026rsquo;t doing much due to family (arrival of my son) and work commitments 17/02/2014 MPhil to PhD transfer report submitted 26/03/2015 Nominated as the School of Systems Engineering candidate for the award of PhD Researcher of the Year 2015 11/04/2016 Thesis submitted 16/06/2016 Viva 08/07/2016 Graduation Part-time challenges and advantages # Like other research students, the university have asked me to provide a brief profile describing my research area, challenges and advantages of part-time study and any advise that other part-time students may find useful. The profile covered the following questions:\nWhat has been your biggest challenge since starting your research? Where do you want to be in 5 years? What were your reasons for choosing part-time study? What are the advantages and challenges of part-time registration? You can find my answers to these questions in this PDF version. Now let\u0026rsquo;s look at some of the key milestones during this journey.\nPhD Proposal # My aim was to explore two fields, the Semantic Web and Cloud Computing. In a nutshell I wanted to solve some of the Semantic Web problems using Cloud Computing. Cloud Computing has unique features such as Elasticity and utility billing that I wanted to explore. Here is my proposal written in 2010, one thing you will notice is that a few things didn\u0026rsquo;t make it to the final thesis. For example I thought I was going to explore OWL (Web Ontology Language), but ended up exploring RDF (Resource Description Framework). Also notice that the title was \u0026lsquo;Scalable Distributed OWL Reasoning for the Semantic Web\u0026rsquo;.\nMPhil/PhD Transfer Report # I had to write a report to transfer from MPhil to PhD. The report is usually written after the first year of full-time study and after 2 years of part-time study. You can find the report here, again notice the title has become \u0026lsquo;Investigating Elastic Cloud Based Reasoning for the Semantic Web\u0026rsquo;. Additionally, a few things didn\u0026rsquo;t actually make it to the final thesis, including using the Recursion Theorem to represent self reproducing Cloud Virtual Machines!. Why did I even think of this? I guess I was inspired by the use of the theorem in modelling viruses, but, as each VM won\u0026rsquo;t likely be acting alone and making copies of itself, swarm intelligence or similar was probably a better alternative to model a fleet of autonomous self managed VMs, but this was a distraction (dead-end), it wasn\u0026rsquo;t what I wanted to do. Moreover, the use of a consistent hash function for workload distribution, which is used in Distributed Hash Tables and NoSQL databases like Amazon DynamoDB. This was replaced with Bin Packing algorithm. As I said, it was a journey to explore and I didn\u0026rsquo;t want to be constraint by anything so I did look at and learnt many things.\nFinal Thesis # I\u0026rsquo;ve submitted my thesis on Monday 11th April 2016, with hopefully a final title of \u0026lsquo;Investigating Elastic Cloud Based RDF Processing\u0026rsquo;. As soon as the viva is done and all is good I will update this section and include a link to the final thesis. In this regard, the word Google occurred 147 times in my thesis, to say I\u0026rsquo;m extremely passionate about Google technologies is an understatement!. Having used so many Google tools (as you will see in the tools section below) and technologies in my research, I felt the least I could do is acknowledge them in my final thesis, here is the extract:\nI would like to thank Google for their great technologies, developer support and documentations. Google support kindly and promptly lifted any quota restrictions that I requested to complete the evaluation of this research. They have kindly provided free credits to use the Google Cloud Platform.\nTools and Books # There are so many tools that I found useful which I will summarise here, Google has the biggest share of these. There is also a corner on my desk that started accumulating a stack of a few books over the last 5 years which were useful for my research. It all started with one book \u0026lsquo;Foundations of Semantic Web Technologies\u0026rsquo;, then a few more books joined the stack. Here is the final stack. In addition to these, I\u0026rsquo;ve a couple of must-read e-books and around 144 journal and conference papers in my Mendeley collection. Note, a few of these books have gathered dust for the last 5 years I thought I would need them, but I didn\u0026rsquo;t!\nThe following free tools made it possible for me to conduct my research, advance the state of the art and writeup my thesis.\nTask Description Tool/Book Getting started How can I get started, what is it like to do a PhD, how can I find a supervisor, etc\u0026hellip;? This book is a must read. How To Get A Phd: A Handbook For Students And Their Supervisors Research methodology Because you can\u0026rsquo;t just hack together a PhD, you need to follow a proper scientific methodology. Plus the PhD is different from say, a technical report. This book is a must read. Design Science Research Methods and Patterns: Innovating Information and Communication Technology Managing research papers A tool to manage, search and collate relevant research papers with the ability to export citations is various formats. Mendeley Desktop Finding research papers A tool that supports advanced search to find relevant research papers. Google Scholar Document management Somewhere to safely manage all the PhD related documents, such that if your hard disk crashed you are not in a crisis. Google Drive Writeup A mechanism to writeup papers and the thesis, as a technologist I\u0026rsquo;ve chosen LaTeX over plain text or word. TeXShop Diagram I needed to produce lots of diagrams, access them anywhere and change them easily. Google Draw Experiments My research experiments were conducted on the cloud, you can find some background on why I went with Google Cloud Platform here. Google Cloud Platform Data gathering and analysis Gathering experiments data and easily analyse it and produce charts. Google Sheets Research Outcome # The biggest initial challenge with part-time research was to keep myself engaged. As a part-time researcher with work and family commitments it’s very easy to get dis-engaged and disconnected from the research. One way to overcome this was to setup targets for me to work towards for example writing a paper for a conference or a journal or taking part in conferences and PhD consortiums. Setting such targets and deadlines enabled me to become very engaged, enthusiastic and connected to my research. I entertained all my colleagues at work by telling them about my research, most importantly, I\u0026rsquo;ve done/taken part in the following to stay engaged.\nOpen Source Contributions # CloudEx (http://cloudex.io), A generic elastic cloud based framework for the execution of embarrassingly parallel tasks**.** ECARF (http://ecarf.io), an Elastic cloud-based RDF triple store for RDF reasoning and RDFS reasoning. Posters # University of Reading postgraduate conference poster competition entry - 01/07/2014 - PDF Presentations # The British Computer Society Doctoral Consortium presentation - 15/05/2014 - PDF University of Reading 3 minutes thesis competition presentation - 01/07/2014 - PDF Computer Science and Electronics Engineering Conference CEEC 2014 presentation - 25/09/2014 - PDF Publications # A Novel Cloud Based Elastic Framework for Big Data Preprocessing, in the Proceeding of the Sixth Computer Science and Electronic Engineering Conference 2014 (CEEC'14), IEEE. O Dawelbeit and R McCrindle - - PDF Efficient Dictionary Compression for Processing RDF Big Data Using Google BigQuery. 2016 IEEE Global Communications Conference: Selected Areas in Communications: Big Data Track (Globecom2016 SAC BD), Washington, USA. O Dawelbeit and R McCrindle - - PDF CloudEx, A Novel Cloud-based Task Execution Framework, 2016 IEEE Global Communications Conference: Workshops: Fifth International Workshop on Cloud Computing Systems, Networks, and Applications (GC16 Workshops CCSNA), Washington, USA, O Dawelbeit and R McCrindle - - PDF It\u0026rsquo;s not over yet # The journey is not over yet, ultimately I\u0026rsquo;m waiting for the viva to take place and hopefully one day I get to use the \u0026lsquo;Dr\u0026rsquo; title and hang the mother of all certificates on the wall. That day, will truly be a day of days. That day, when it comes, if it comes, will be worth all the hard work and the difficult moments I lived through over the past 5 years, but until then I quote this feeling of \u0026lsquo;Euphoria\u0026rsquo; from How To Get a PhD book:\nAfter submission of the thesis there is a period of anxiety and expectation that you have to live through waiting for the day of the viva. There is then, when you are no longer constantly confronting your thesis, the feeling of a gap in your life – a burden that has been lifted from your shoulders. Those feelings are mitigated, however, by the knowledge that all is not yet over.\nUpdate: 15/07/2016\nOn the positive side, there is the energetic, the positive and the kind who feels happy when they see others succeed. Also, POSTD, a Japanese news website focused on IT trends and programming tips, have translated this article to Japanese \u0026ldquo;私がどのようにして、ソフトウェア開発とオープンソースへの情熱をパートタイムで博士号を取得することで満たしたか\u0026rdquo;.\nReflections on Thesis Writing # The most challenging activity for me during this journey was writing. I felt comfortable tackling challenging problems, running experiments, writing code and analysing data, writing however was difficult. Sometimes I was very productive and wrote many pages in a day and other times struggled to put a paragraph together in an hour. I found that the source of the problem of not being able to write is the fact that I don\u0026rsquo;t have the argument formulated in my head. So trying to write whilst going through the thought process of formulating an argument was time consuming. I found it was easy to think, formulate the argument, visualise it, then finally write it up, this approach has worked really well for me.\nThe Thought Process # To go through the thought process and visualise my arguments before writing them up, my supervisor suggested Post-it notes. So I did Post-it notes and put them up on the wall, but I felt this technique was rigid, I needed a fluid brainstorming approach. Ideas were changing all the time and I needed more space and colours to express my thoughts. Post-it notes weren\u0026rsquo;t sufficient, I needed to draw, write, use different colours, wipe and try again.\nWelcome the Whiteboard # Luckily I had a whiteboard and ordered some colourful whiteboard markers then started jotting down my thoughts on the whiteboard using different colours, I then took a picture using the Microsoft Office Lens app, which has a whiteboard option. Finally I printed the image so I had a paper record that I could annotate further, this also enabled me to wipe the whiteboard immediately for more brainstorming.\nHaving the Courage to Take Time off Work # Also luckily for me, my supervisor encouraged me to write journal and conference papers all the way through. Although some didn\u0026rsquo;t get published, what I\u0026rsquo;ve written up over the years was a valuable asset when I started writing up the thesis. End of July last year (2015), I took two month off work to focus on running experiments and analysing the data. Then I was back to work in October 2015 and started writing up the thesis. It was a slow process, I\u0026rsquo;ve completed one chapter by the end of November. It was clear that work was getting in the way of completing the thesis, so I\u0026rsquo;ve decided to take time off work from the end of December 2015. I worked hard writing up the thesis over the following 3 months, sometimes doing up to 14 hours per day. It wasn\u0026rsquo;t healthy, but had to be done and it was rewarding to be able to produce one chapter after the other.\nDealing with Boredom # Although it was rewarding to produce chapters, no two will disagree that writing up a thesis can become a very boring activity. This was true for me, at times I was so bored that I had to do something else as well. I spend time writing blog posts, doing various talks in local meetups and playing the occasional Splatoon on the Nintendo Wii U with my children. It was a good laugh and got me to spend time with the kids during the tedious writing up activity. The little ones were overly happy when I helped them defeat the big baddy DJ Octavio, although I ended up with sore thumbs . Doing these activities I found I was very fresh when I went back to thesis writing.\nThe Viva, Corrections, Graduation and Beyond # Waiting Forever # By the beginning of June, almost two months after submitting my thesis, there was no news regarding when the Viva was going to be. I was getting very anxious because a late Viva meant I would miss the July graduation and instead wait until December and that is a long time to wait for the impatient. Even if the Viva was decided sometime in June, it was still very tight to make it for the 8th July graduation, there were various deadlines that had to be met. I sent my supervisor a few emails to enquire, but here was no response, so I thought instead of getting stressed I should just forget about the Viva for the time being. Then on the 9th June my supervisor emails me to say the Viva has been scheduled for the 16th June, what a joy .\nThe period leading up to the Viva was one of hope and anxiety, but my wife kept telling me not worry and that I would be fine. To be honest the PhD journey has gone for long enough and all what I wanted at that stage was to conclude the hard work I\u0026rsquo;ve put in this process over the last few years and move on with my life. I had two weekends before the big day, which I spent reading my thesis cover to cover, taking notes, coming up with hypothetical questions. The \u0026lsquo;Surviving the Viva\u0026rsquo; university guide was a good read.\nThe Viva - Day of Days # It\u0026rsquo;s the Viva day, I\u0026rsquo;m fresh and up very early, I made my way to the university at 9:30am for 10:00am start. I met my internal and external examiners and spent the following two hours defending my thesis. The Viva is over, I\u0026rsquo;m asked to leave the room, I find my wife outside waiting for me with a smile, repeating the same statement she has been saying to me all the time, \u0026lsquo;you will be fine\u0026rsquo;. After a short while I get called back into the room for the good news that I\u0026rsquo;ve passed with minor corrections, joy and celebrations all around.\nThe End is Almost in Sight # Still, time was running out, if I wanted to graduate in July I had to make my corrections and they needed to be accepted by the examiners and then available at the exam office before the 24th June. My internal examiner and supervisor were very understanding that I needed to meet this deadline. The Viva was on Thursday, on Friday I received a list of corrections, then I spent the weekend, working 14 hours each day to complete the corrections. This was painful, but had to be done, the end was almost in sight, just that final push, PUSH, PUSH. Late on Sunday night, the corrections were coming to an end, I emailed the corrections to the examiners, on Monday morning I got asked to do some more minor amendments, which I did. Finally on the same day I got a confirmation email from the exam office that my corrections were accepted and I\u0026rsquo;m to be awarded the PhD from 30th June, what a joy!. I had three copies of my hard-bound thesis submitted to the exam office. That was it, I waited impatiently for the graduation, preparing and trying out my best suit.\nGraduation Day # Finally, after 5 years comes this day. It was truly a day of days that words can\u0026rsquo;t explain and do justice. This picture captured the atmosphere on the day.\nFinally # Whatever is your endeavour, if you put effort into it, anything can be done. Yes You Can!\nFollow @omerio\n","date":"15 May 2016","externalUrl":null,"permalink":"/how-i-satisfied-my-passion-for-software-development-and-open-source-by-doing-a-part-time-phd/","section":"Technology with Passion","summary":"A version of this article was translated to Japanese.","title":"How I Satisfied My Passion for Software Development and Open-Source by Doing a Part-Time PhD","type":"page"},{"content":"","date":"16 January 2016","externalUrl":null,"permalink":"/categories/cloud/","section":"Categories","summary":"","title":"Cloud","type":"categories"},{"content":"","date":"16 January 2016","externalUrl":null,"permalink":"/tags/google-app-engine/","section":"Tags","summary":"","title":"Google App Engine","type":"tags"},{"content":"","date":"16 January 2016","externalUrl":null,"permalink":"/tags/google-charts/","section":"Tags","summary":"","title":"Google Charts","type":"tags"},{"content":"","date":"16 January 2016","externalUrl":null,"permalink":"/categories/iot/","section":"Categories","summary":"","title":"IoT","type":"categories"},{"content":"","date":"16 January 2016","externalUrl":null,"permalink":"/categories/java/","section":"Categories","summary":"","title":"Java","type":"categories"},{"content":"","date":"16 January 2016","externalUrl":null,"permalink":"/tags/java/","section":"Tags","summary":"","title":"Java","type":"tags"},{"content":"","date":"16 January 2016","externalUrl":null,"permalink":"/categories/kubernetes/","section":"Categories","summary":"","title":"Kubernetes","type":"categories"},{"content":"","date":"16 January 2016","externalUrl":null,"permalink":"/tags/pi4j/","section":"Tags","summary":"","title":"Pi4J","type":"tags"},{"content":"","date":"16 January 2016","externalUrl":null,"permalink":"/tags/raspberry-pi-zero/","section":"Tags","summary":"","title":"Raspberry Pi Zero","type":"tags"},{"content":" Introduction # In this article I\u0026rsquo;m going to explain how you can setup a real time sensors dashboard using Google App Engine and a Raspberry Pi Zero. Here is a live demo and the video below shows the end result. If this sounds interesting, then read on for the details.\nGoogle App Engine is a Platform as a Service (PaaS) that lets you deploy and run your applications on the Google infrastructure without having to worry about setting up your own hardware, Operating System or server. We will also use Google Charts — free, powerful and simple charting tools — to plot the sensors data into line charts. We have also used Initializr — a HTML5 templates generator — to generate a template for the dashboard which includes Bootstrap, jQuery and other useful frontend resources.\nFor the hardware we will use the glorious Raspberry Pi Zero, a fully fledged computer smaller than a credit card. The Pi Zero features a 1Ghz single-core CPU, 512MB RAM, mini HDMI and USB and a 40 pin GPIO header. We will connect a few sensors to the GPIO pins and send their data over to Google App Engine. When user views the dashboard both the values and charts on the dashboard update in real time whenever new data arrives from the sensors. The source code for both the App Engine dashboard and the Pi Zero app is in Github with instructions on how to build and deploy each project.\nArchitecture Overview # In this article we use Java as the programming language, Java is supported by both Google App Engine and the Pi Zero through the Pi4J library. However, if you prefer you can easily rework the code into Python, which is also supported by both the Pi Zero and Google App Engine. The new versions of the Pi Zero Operating System, Raspbian, comes with Oracle Java 8 pre-installed. Ultimately we deploy and run an executable JAR on the Pi Zero, this JAR reads the input from the sensors and updates Google App Engine. We use Apache Maven to compile and build the code on the Pi Zero, although this is not necessary as you can build the code on your laptop and copy the JAR over to the Pi Zero.\nThe full architecture is shown below:\nOn the Google App Engine side we use Cloud Endpoint, a very powerful service that we can use to create a backend API and client libraries for mobile and web by using annotations. Since the generated Android client is Java based we can use it with the Pi Zero application. The API that the Pi Zero calls is authenticated using Google OAuth 2.0 for installed applications. We use an authenticated API to ensure that only our authorised Pi Zero is actually able to send updates to the server. We also use the Channel API, one of the Google App Engine services that enables us to establish a persistent connection between the browser and the server so we can push real time messages to the browser. This is obviously better than continuously polling the server for updates.\nThe Hardware # The hardware we are building in this article is basic and is inspired from other online articles. Instead of repeating the setup here I will reference these articles where appropriate. Here is the Breadboard view of the completed circuit generated using Fritzing. If you install Fritzing you can download this schematic here:\nI need help with the hardware # For the hardware used in this article, my friend Lez has created a bare board and a populated board which can be purchased on his Website. You can use the bare board instead of a breadboard, source and solder the components yourself. The populated board comes with all the components apart from the solar panel and the photocell (LDR), which are mentioned below and can be bought from Amazon.\nI want to assemble the hardware myself # The purpose of this hardware is to provide the readings of three sensors, voltage from a solar cell, temperature from an analogue temperature sensor and illuminance (LUX) using a photocell. Because the output of the temperature sensor and photocell is analogue and the Pi Zero only accepts digital input, we use an Analogue to Digital Converter (ADC).\nHere are the components used:\nResisters: 3 x 220Ω, 1 x 10KΩ, 1 x 100KΩ and 1 x 150KΩ General purpose diode (for the solar cell) Three LEDs (red, yellow and green) Generic photocell Analog temperature sensor: Adafruit TMP36 Mini solar cell 6.5 x 6.5cm 5.5v 90mA 0.6W Analogue to Digital Converter: Adafruit MCP3008 8-Channel 10-Bit ADC With SPI Interface Raspberry Pi 40 pins coupler and cable Breadboard Raspberry Pi Zero with official WiFi dongle and adapter kit. If you use the Pi Zero then you need to solder pins on the unpopulated GPIO headers. We use the 3.3v pin 1 on the Pi Zero to power our various components. We use a red LED connected to the Pi\u0026rsquo;s 3.3v pin as an indication of power, a green LED as a load for the solar cell and a yellow LED to indicate when the Pi is saving data to the cloud. The yellow LED is connected to pin 18 on the Pi Zero. All LEDs are connected to 220Ω resistors.\nConnecting the Sensors # Since all the three components, temperature sensor, photocell and solar cell output analogue voltage between 0 and 3.3v, we use the MCP3008 ADC to convert these analogue voltage values into digital values between 0 and 1023. This is because the MCP3008 is a 10 bit ADC and the largest decimal number you can represent with 10 bits is \\(2^{10} - 1 = 1023\\). The MCP3008 ADC has 8 input channels, which means we can connect up to 8 analogue inputs to it. The MCP3008 uses the Serial Peripheral Interface (SPI) for communication, which is already supported by the Pi Zero. Bear in mind to program the MCP3008 ADC we need to understand the structure of the data that should be sent to it, these are explained on the datasheet on page 21. The Pi Zero code is full of comments that explains why particular values are being sent to the ADC.\nWe connect the photocell to channel 0, the temperature sensor to channel 1 and the solar cell to channel 2, this leaves the other 5 channels free. The photocell and temperature sensor connections used here are the same as in this raspberrypi-spy article, so we won\u0026rsquo;t repeat this setup here. Be sure to follow this article on the same website to enable the SPI interfaces on your Pi.\nTemperature # The temperature formula we use for the TMP36 is based on the one on this article on the Adafruit website:\nmillivolts = read_adc0 * ( 3300.0 / 1023.0 ) temp_C = ((millivolts - 100.0) / 10.0) - 40.0 Illuminance # Again, to measure the illuminance we use a lookup table based on these values on the Adafruit website, bear in mind these are just an approximation because we are using a generic photocell without the datasheet. If you want more accurate LUX readings then it\u0026rsquo;s better to use a digital LUX sensor.\nVoltage # For the solar cell a diode is connected to its positive terminal as described here. Additionally, because the cell can produce a maximum of 5.5v we use a voltage divider to ensure the ADC receives only 3.3v when the cell output is at its maximum (5.5v). The circuit on the left show the solar cell with resister R2 = 220Ω and Green LED3 as load. We then use R6 and R7 as voltage divider so that we can calculate the cell output voltage \\(V_{in}\\) from the voltage read by the ADC \\(V_{out}\\) using this formula:\n\\[ V_{in} = \\frac{V_{out} * (R_6+R_7)}{R_7} \\] This circuit for measuring the solar cell voltage is inspired by this article, which includes the wiring and the formula for measuring the current as well. This article on RasPi.TV for measuring the voltage output of a 2 cell lithium polymer (lipo) battery is also useful. Here is the fully assembled Breadboard and Pi Zero:\nThe Dashboard App # The dashboard app is a simple Java Web application that is deployed to Google App Engine. The source code is located in Github. Here we will provide a brief review of each of the key files. The instructions on how to deploy the application are on Github, simply follow them to get your own dashboard up and running on Google App Engine.\nThe Java classes are organised in three packages, the endpoint package contains the sensor data API class. The entities package contains the objects that we persist to the Datastore (the App Engine NoSQL database). The servlets package contains the Java web application servlets. The most important class is the SensorDataEndpoint.java that implements the sensor data API.\nThe Sensor Data API # The sensor data API is coded using Cloud Endpoints and implemented in SensorDataEndpoint.java shown below. As you can see we have created a fully fledged API by using annotations such as @Api on line 23 and @ApiMethod on line 37. Google App Engine will expose the create method annotated with @ApiMethod as a REST/RPC API that uses JSON over HTTP. The client libraries for Android and iOS can also be created by the user.\nWhen the Pi Zero calls this create method, Google App Engine will populate and inject a User object if the Cloud Endpoint request was authenticated using a valid Google Account. If not, we throw some exceptions, otherwise we go ahead and save the SensorData object. Note that App Engine is doing the marshalling and unmarshalling of the SensorData object from JSON to Java and vice versa. Next step on line 55 we use the App Engine Channel API to notify any connected clients (browsers) that we have new data. We retrieve all active clients from the Datastore and update each of their channels.\nHTTP Requests Handlers # When a user loads the dashboard into their browser the JavaScript makes an AJAX call to /getdata, this is mapped to the StartServlet.java.\nThis servlet will use the HTTP session ID to determine if the user has a Client entity saved in the datastore, which mean they already have an open channel. If not, then a new Client entity and a channel are created, the HTTP session ID is used to generate a channel token, which is sent to the client. The JavaScript uses this token to establish a channel connection and registers a handler which is invoked when messages are received. The servlet then retrieves the sensor data for the last X minutes and sends a response back to the client as JSON.\nThe ChannelHandler.java class handles channels connection and disconnection, whenever a new client connects to or disconnects from a channel this handler is invoked. When a client disconnects we use this handler to remove their corresponding Client entity from the datastore, so we don\u0026rsquo;t have to worry about updating their channel anymore.\nThe Frontend Code # The bulk of the frontend code is in main.js, in here we make an AJAX call to get the channel token and a list of sensor data for the last X minutes. Once we get the data from App Engine we reformat it so that it\u0026rsquo;s suitable for the Google Line Chart. We then use the channel token to establish a channel connection and register an event handler for when channel messages are received.\nThe user interface offers the user three charts with the ability to retrieve data for the last minute, hour, day and 30 days. The user can also choose to auto refresh the charts when new sensor data is received through the Channel API. We use the jQuery Animate Number plugin to add a nice effect when new sensor values are received.\nThe Raspberry Pi App # The Raspberry Pi app is a Java JAR with an executable main class. The source code for the app is in GitHub. The app consists of a bunch of Java files, the files highlighted in red are the ones we\u0026rsquo;ve created. The other classes are automatically generated by the App Engine Cloud Endpoints utility and we just copied and pasted them in the app. These classes represent the Cloud Endpoints client that the Pi is going to call when it\u0026rsquo;s sending sensor data to App Engine.\nThe CmdLineAuthenticationProvider.java file provides the implementation for an OAuth 2.0 installed client. When the application is started from the command-line for the first time it asks the user to type in a URL into their browser:\n$ java -jar RaspberryPiApp.jar Please open the following address in your browser: https://accounts.google.com/o/oauth2/auth?client_id=111109056482-6jn34uijapemnbfk4k7ukcv7eclse1ln.apps.googleusercontent.com\u0026amp;redirect_uri=urn:ietf:wg:oauth:2.0:oob\u0026amp;response_type=code\u0026amp;scope=https://www.googleapis.com/auth/userinfo.email Attempting to open that address in the default browser now... Please enter code: afdsadsfasdfawikehUzIMoSTk9hZ96VkFMbZA0 Once the URL is opened on the browser, it will forward to the Google accounts \u0026lsquo;Request for permissions\u0026rsquo; page. We need to be signed into a Google account and accept the request. A short code will be generated which we then need to copy and paste into our Pi app as shown in \u0026lsquo;Please enter code:\u0026rsquo; snippet above. Once that is done, the application will now exchange this code for an OAuth 2.0 token that can be used to make requests on our behalf.\nThe RaspberryPiApp.java is where everything else apart from authentication takes place. Here the values of the sensors are read and sent over to Google App Engine using Cloud Endpoints. The two noteworthy methods are shown below. readChannel, sends 3 bytes to the MCP3008 chip using the SPI interface and gets back 3 bytes, from which we extract 10 bits. The logic is explained in the comments below, uses Java bitwise operators and is based on the Python code in this article. The other sendSensorData method uses the Cloud Endpoints clients to send the sensor data over to Google App Engine.\nSummary # In this article we have presented a full working solution for implementing a real time sensors dashboard (https://raspberrypi-dash.appspot.com/) on Google App Engine Platform as a Service. We have used the following Google App Engine Technologies:\nCloud Endpoints Channel API Datastore We have also used Google OAuth 2.0 to authenticate updates from our Raspberry Pi Zero. On the frontend side we have used Bootstrap, Google Charts and jQuery. For the Raspberry Pi, we have tested this app on Pi Zero and used the Pi4J library to be able to interact with the GPIO pins using Java.\nWhere to Next # You can deploy and run both applications as detailed in GitHub, you can also modify both the dashboard and Pi code to suit your need:\nApp Engine Dashboard: https://github.com/omerio/raspberrypi-appengine-portal Raspberry Pi App: https://github.com/omerio/raspberrypi-app If you are interested in implementing Internet of Things (IoT) solutions on the Google Cloud Platform, or wondering about the best architecture to follow then make sure to checkout the IoT documentations.\nFollow @omerio\n","date":"16 January 2016","externalUrl":null,"permalink":"/2016/01/16/real-time-sensor-dashboard-using-google-app-engine-and-raspberry-pi-zero/","section":"Blog","summary":"In this article I’m going to explain how you can setup a real time sensors dashboard using Google App Engine and a Raspberry Pi Zero. Here is a live demo and the video below shows the end result. If this sounds…","title":"Real Time Sensor Dashboard Using Google App Engine and Raspberry Pi Zero","type":"posts"},{"content":"","date":"18 December 2015","externalUrl":null,"permalink":"/tags/containers/","section":"Tags","summary":"","title":"Containers","type":"tags"},{"content":"","date":"18 December 2015","externalUrl":null,"permalink":"/categories/infrastructure/","section":"Categories","summary":"","title":"Infrastructure","type":"categories"},{"content":"","date":"18 December 2015","externalUrl":null,"permalink":"/tags/kubernetes/","section":"Tags","summary":"","title":"Kubernetes","type":"tags"},{"content":" This post was written in December 2015 for Kubernetes 1.1. The core concepts (pods, labels, services, replication) are still accurate, but ReplicationControllers have since been superseded by Deployments and ReplicaSets, and the commands and Google Container Engine screenshots are out of date. TL;DR # In this post I will provide a brief explanation of the key concepts of Kubernetes. I will avoid using lengthy definitions, these are already available in the Kubernetes documentations. Rather, I will be using a few diagrams (some animated) and examples to explain these concepts. I found a few of the concepts difficult to fully grasp without a diagram (Service for example). Where appropriate I will also provide links to the Kubernetes documentations if you want to deep dive.\nLet’s start the clock.\nWhat is Kubernetes? # Kubernetes (k8s) is an open source platform for automating container operations such as deployment, scheduling and scalability across a cluster of nodes. If you have ever used Docker container technology to deploy your containers, then think of Docker as a low level component used internally by Kubernetes to deploy containers. Not just Docker, but Kubernetes also supports Rocket, another container technology.\nKubernetes orchestrates your containers so together they are performing a Symphony. This could be anything from, but not limited to, the following:\nautomate the deployment and replication of containers, scale in or out containers on the fly, organise containers in groups and provide load balancing between them, easily roll out new versions of application containers, provide container resilience, if a container dies it gets replaced, etc.. In fact, with Kubernetes you can deploy a full cluster of multi-tiered containers (frontend, backend, etc…) with a single configuration file and a single command:\n$ kubectl create -f single-config-file.yaml kubectl is a command-line program for interacting with the Kubernetes API. Now let’s introduce some of the key concepts.\nCluster # A cluster is a group of nodes, they can be physical servers or virtual machines that has the Kubernetes platform installed. The diagram below is an illustration of such cluster. Note this diagram is very simplified to highlight the key concepts. For a typical Kubernetes architecture diagram see here.\nView Google Drawings (Feel free to copy \u0026amp; reuse)\nLooking at the diagram you can spot the following components, I used icons to represent Service \u0026amp; Label:\nPods Containers Label(s) () Replication Controllers Service () Nodes Kubernetes Master I will start off with Pods because they are the smallest deployable units in Kubernetes that can be created scheduled and managed. It’s all about the Pods, if you deploy a single container it will be deployed in its own Pod.\nPods # Pods (green boxes) are scheduled to Nodes and contain a group of co-located Containers and Volumes. Containers in the same Pod share the same network namespace and can communicate with each other using localhost. Pods are considered to be ephemeral rather than durable entities. You might be asking yourself a few questions:\nIf Pods are ephemeral how can I persist my container data across container restarts? Well, Kubernetes supports the concept of Volumes so you can use a Volume type that is persistent.\nDo I create Pods manually, what if I want to create a few copies of the same container do I have to create each one individually? You can create individual Pods manually, but you can use a Replication Controller to rollout multiple copies using a Pod template, which will be explained below.\nIf Pods are ephemeral and their IP address might change if they get restarted how can I reliability reference my backend container from a frontend container? In this case you will use a Service as explained below.\nBefore I go on to explain Replication Controllers and Services, let me first introduce Labels.\nLabels # As you can see from the diagram, some of the Pods have labels (). A Label is a key/value pair attached to Pods and convey user-defined attributes. For example you might create a ‘tier’ and an ‘app’ tags to tag your containers by applying the Labels (tier=frontend, app=myapp) to your frontend Pods and Labels (tier=backend, app=myapp) to backend Pods. You can then use Selectors to select Pods with particular Labels and apply Services or Replication Controllers to them.\nReplication Controllers # Do I create Pods manually, what if I want to create a few copies of the same Pod, do I have to create each one individually, can I group Pods into logical groups?\nReplication Controllers ensure the specified number of Pod “replicas” are running at any one time. If you created a Replication Controller for a Pod and specified 3 replicas, it will create 3 Pods and will continuously monitor them. If one Pod dies then the Replication Controller will replace it to maintain a total count of 3. This is illustrated in the animated image below:\nView Google Drawings (Feel free to copy \u0026amp; reuse)\nIf the Pod that died comes back then you have 4 Pods, consequently the Replication Controller will terminate one so the total count is 3. If you change the number of replicas to 5 on the fly, the Replication Controller will immediately start 2 new Pods so the total count is 5. You can also scale down Pods this way, a handy feature performing rolling updates.\nWhen creating a Replication Controller you need to specify two things:\nPod Template: the template that will be used to create the Pods replicas. Labels: the labels for the Pods that this Replication Controller should monitor. So now you have created a few replicas of a Pod, how do you load balance between them? Enter Services.\nServices # If Pods are ephemeral and their IP address might change if they get restarted how can I reliably reference my backend container from a frontend container?\nService is an abstraction that defines a set of Pods and a policy to access them. Services find their group of Pods using Labels. Because Services are abstraction you don\u0026rsquo;t usually see them in diagrams which makes the concept hard to understand.\nNow, imagine you have 2 backend Pods and you defined a backend Service named ‘backend-service’ with label selector (tier=backend, app=myapp). Service backend-service will facilitate two key things:\nA cluster-local DNS entry will be created for the Service so your frontend Pod only need to do a DNS lookup for hostname ‘backend-service’ this will resolve to a stable IP address that your frontend application can use.\nSo now your frontend has got an IP address for the backend-service, but which one of the 2 backend Pods will it access? The Service will provide transparent load balancing between the 2 backend Pods and forward the request to any one of them (see the animated diagram below). This is done by using a proxy (kube-proxy) that runs on each Node. More technical details here.\nThis animated diagram illustrates the function of Services. Note that this diagram is overly simplified. The underlying networking and routing involved to achieve this transparent load balancing is relatively advanced if you are not into network configurations. Have a peek here if you are interested in a deep dive.\nView Google Drawings (Feel free to copy \u0026amp; reuse)\nThere is a special type of Kubernetes Services called ‘LoadBalancer’, which is used as an external load balancer to balance traffic between a number of Pods. Handy for load balancing Web traffic for example.\nNodes # A node (the orange box) is a physical or virtual machine that acts as a Kubernetes worker, used to be called Minion. Each node runs the following key Kubernetes components:\nKubelet: is the primary node agent. kube-proxy: used by Services to proxy connections to Pods as explained above. Docker or Rocket. The container technology that Kubernetes uses to create containers. Kubernetes Master # The cluster has a Kubernetes Master (the box in purple). The Kubernetes Master provides a unified view into the cluster and has a number of components such the Kubernetes API Server. The API Server provides a REST endpoint that can be used to interact with the cluster. The master also includes the Replication Controllers used to create and replicate Pods.\nWhere to next # Now armed with the basic knowledge of the key concepts of Kubernetes you can have a look at my recent follow on post \u0026ldquo;Getting Started with Kubernetes on Google Container Engine\u0026rdquo;. You can also have a look at the Kubernetes user guide. The user guide provides a quick walkthrough and a thorough walkthrough.\nIf you can’t wait and want to get your hands on some Kubernetes practice then head over to Google Container Engine. Google Container Engine is a hosted Kubernetes container environment. Simply signup/signin and run through the Walkthroughs examples.\nFollow @omerio\nRecommended Tutorial # ","date":"18 December 2015","externalUrl":null,"permalink":"/2015/12/18/learn-the-kubernetes-key-concepts-in-10-minutes/","section":"Blog","summary":"In this post I will provide a brief explanation of the key concepts of Kubernetes. I will avoid using lengthy definitions, these are already available in the Kubernetes documentations. Rather, I will be using a few…","title":"Learn the Kubernetes Key Concepts in 10 Minutes","type":"posts"},{"content":"This tiny beauty has arrived through my letterbox today. The glorious Raspberry Pi Zero is a fully fledged computer smaller than a credit card. With 1Ghz single-core CPU, 512MB RAM, mini HDMI and USB and a 40 pin GPIO header you won\u0026rsquo;t run out of ideas of what to do with it. People have created all sort of cool projects using Raspberry Pis. From running container technology like Kubernetes on a Raspberry Pi cluster, to running good old Retro Games on old TV sets.\nWhy Now? # To be honest this is my first Raspberry Pi. I wasn\u0026rsquo;t interested in acquiring any of the other Pi models, mainly because of the price and I will explain why. For me a Raspberry Pi is for connecting and experimenting with external electronic modules through the GPIO pins rather than just using it as a mere computer. In particular connecting actuators like stepper motors and sensors. These sensors can read real-world quantities (temperature, humidity, motion, etc\u0026hellip;) and feed raw data to the Pi. The software running on the Pi can then pre-process this raw data before sending it to the Cloud (does this sound like IoT?). To do this, most likely you will be connecting and playing with electronic components, hence running the risk of burning the precious Pi. Small mistakes, like not properly grounding a circuit, can result in some smoke and $20 - $35 burnt. Now the $5 Pi Zero is a game changer, with postage costing a sizeable fraction of its price you are better off ordering 10 together!. And it doesn\u0026rsquo;t matter if you burn or misplace one or two while experimenting, it\u0026rsquo;s only $5, you can just get a replacement. I believe it will give a massive boast to prototyping IoT ideas.\nEnter the Late 90s # I started off my formal education studying Electronics Engineering, before realising that my passion for software was greater and decided to completely switch to Software. It has been years since I last set my hands on a hardware/software interface project.\nThis 386 Packard Bell Legend form the early 90s was my Raspberry Pi Zero equivalent in 1997!. And I\u0026rsquo;m not joking, the model was called \u0026lsquo;Legend\u0026rsquo; and \u0026lsquo;Elite\u0026rsquo; as well. When it was new it probably sold for more than 200 times the Pi Zero price.\nI\u0026rsquo;m expecting a few objections for comparing this monstrous piece of old hardware to the elegant and tiny Pi Zero. After all it\u0026rsquo;s more than a 1000 times heavier and 30 times slower . Bear with me, if we put size aside, and I know it\u0026rsquo;s a big ask, and focus instead on capabilities and price. Your dreams of getting a full computer in 1997 with reasonable speed (33Mhz or above) for $5 or less was only going to come true with a 386. During that time people were upgrading to Pentium MMX PCs and getting rid of their old 386s and 486s. It was easy back then getting a 386 almost for free, so if you ended up burning it, getting a replacement wasn\u0026rsquo;t a big deal.\nThat 386 was more than sufficient for small projects involving external electronic circuits. At 33Mhz and 8MB of RAM it can comfortably run DOS and Turbo Pascal. Enough to write code that interacts with an external circuit connected through the Parallel Port (used for connecting printers back then). Activating the port though and sending data to it needed Assembly Language and Bitwise operations knowledge. I found some old Turbo Pascal Units I\u0026rsquo;ve written to interact with the Parallel Port and it\u0026rsquo;s pretty low-level. Moving data between registers and invoking interrupts:\nAnd here is an example of the circuit schematics I enjoyed drawing. It was simple stuff, but was great fun for filling up spare time:\nSize Does Matter # Coming back to size, I was really passionate about these software/hardware interface projects and had a few ideas of adding the Internet to the mix. But having a bulky 386 pumping heat under my feet in a small flat with wires sticking out in a non child friendly way wasn\u0026rsquo;t going to last for long. This was the reason I gave up on these projects, but I kept the kit hoping for a return one day. Now the Pi Zero ticks all the boxes: size, capabilities and prices. With a Linux flavoured OS and a high level GPIO Python library, I\u0026rsquo;m hoping to make a come back in 2016 with some IoT project ideas.\nHave you been in a similar situation and hoping for a come back on a Pi, or have you been using a Pi recently? Would love to hear your thoughts and experiences.\n","date":"14 December 2015","externalUrl":null,"permalink":"/2015/12/14/my-1997-equivalent-of-raspberry-pi-zero/","section":"Blog","summary":"This tiny beauty has arrived through my letterbox today. The glorious Raspberry Pi Zero is a fully fledged computer smaller than a credit card. With 1Ghz single-core CPU, 512MB RAM, mini HDMI and USB and a 40 pin GPIO…","title":"My 1997 Equivalent of Raspberry Pi Zero","type":"posts"},{"content":"","date":"20 November 2014","externalUrl":null,"permalink":"/categories/mobile/","section":"Categories","summary":"","title":"Mobile","type":"categories"},{"content":"Following on from my previous post, in this post I rate the various platforms (iOS, Android, Windows Phone and BlackBerry) based on my own experience on their cross platform development ROI. I specifically use to the following key measures I\u0026rsquo;ve mentioned in the previous post as a basis for my rating:\nThe ability to re-use cross platform code developed previously for new projects. The ability to leverage the knowledge acquired without having to constantly learn new knowledge. The ease of the market deployment process. Project MyWorldApp # Three years ago I embarked on a personal project codenamed MyWorldApp to keep me busy in the evening and weekends and to further strengthen my knowledge in technologies like Spring, Spring Roo and cross platform mobile development using PhoneGap and Sencha Touch. MyWorldApp is a framework for cross platform mobile apps development that is backed by a multi-tenancy Content Management System (CMS). Anyway, I\u0026rsquo;ve completed the project two years ago (2012) by setting up a rapid workflow for cross platform mobile development with custom branding and mothballed it.\nRecently I needed to develop a cross platform mobile application for OfficeMA Timesheet (one of my many projects). OfficeMA uses ExtJS JavaScript library for the user interface (frontend), so theoretically I should be able to leverage the same code-base on a cross platform mobile app and indeed this was the case. And having the cross platform capability developed as part of my work on MyWorldApp I was in a winning situation, I should be able to leverage the work done previously and maximise my return on the investment of developing MyWorldApp. This is a common scenario with many companies that have an in-house cross platform capability, you expect to leverage the work done for one app in order to develop another. So here is my experience of trying to re-use the cross platform capability developed two years ago.\nThe Good, the Bad and the Ugly # The Good # Android was the best, I\u0026rsquo;ve restored the old code base, update with the new frontend code and deployed to a simulator and the application ran fine, then tested on a device and it was all hunky dory. I then headed over to Google Play to deploy the app and everything was a breathe. The developer console was crystal clear, all the fields have help text and instructions clearly highlighting what is needed. So all in all it took less than a day to integrate the common code base, test and deploy the app to Google Play.\nThe Bad # The iOS (iPhone/iPad) landscape has changed a great deal between iOS 5, 6 \u0026amp; 7. A number of APIs which were public are made private, the layout of the tools (XCode) kept changing between versions, the user interface layout on the phone is changed (i.e. signal bar is now part of the view). All of this meant the code that was developed two years ago didn\u0026rsquo;t work. It needed to constantly evolve to keep up with all the changes to iOS.\nThe code is now broken and a series of tools/frameworks needed to be updated to cater for newer versions of iOS, which means spending time learning new things. All in all it took almost a whole week to get the app working on iOS. And what was supposed to be cross platform development turned into native development with deep dives into Objective C code to find what was going wrong, this will be a major challenge if your cross platform developers are primarily frontend developers (JavaScript, HTML, CSS).\nThe deployment to the App Store wasn\u0026rsquo;t less painful either, with random errors, out of date documentations and two rounds of updates and rejections it took around a month to get the application to the store!. All of this mean your cross platform development capability needs to stay up to date with iOS changes, otherwise it\u0026rsquo;s likely that each app development cycle will put a drain on your resources.\nThe Ugly # Two years ago when I looked at Windows Phone 7, Internet Explorer 9 did not support any touch events, a core feature required for any JavaScript apps that use the Web view. Adding to this obstacle the fact that IE, unlike iOS, Android and BlackBerry browsers, doesn\u0026rsquo;t use the Apple WebKit engine, you end up developing custom JavaScript/CSS code to cater for the differences. It seems only recently that touch events are added to Internet Explorer 11 running on Window Phone 8.1. It will be very difficult to get your common cross platform code to work in Window Phone without major and tricky changes. It would be easier to start fresh using the platform\u0026rsquo;s native programming language and SDK.\nMoving on to BlackBerry, two years ago BlackBerry OS 6 and 7 were the predominant versions with a range of devices such as BlackBerry Bold and Torch. The BlackBerry platform at the time was undergoing major changes and updates moving from Java to the WebWorks HTML5 framework. At the time the BlackBerry devices struggled a great deal running cross platform mobile apps and showed poor performance compared to iOS and Android devices. Two years on, the BlackBerry landscape has completely changed with the introduction of the BlackBerry 10 and a range of new devices. With less than 1.3% market share and a changing landscape it will be very difficult to maintain a cross platform capability for BlackBerry, not only that your developers will need to learn new skills, but you also need to invest in new devices.\nI would love to hear your thoughts or your experience with cross platform mobile development.\nNote: I\u0026rsquo;m not a mobile developer, it\u0026rsquo;s just that cross platform mobile development should be easy for anyone with basic Web development experience.\n","date":"20 November 2014","externalUrl":null,"permalink":"/2014/11/20/the-good-the-bad-and-the-ugly-of-cross-platform-mobile-development/","section":"Blog","summary":"Following on from my previous post, in this post I rate the various platforms (iOS, Android, Windows Phone and BlackBerry) based on my own experience on their cross platform development ROI. I specifically use to the…","title":"The Good, the Bad and the Ugly of Cross Platform Mobile Development","type":"posts"},{"content":"When it comes to job interviews I\u0026rsquo;ve had my share throughout the years being on both sides of the table. I\u0026rsquo;ve conducted my first mock interview years back when my employer at the time sent me on a City \u0026amp; Guilds Recruitment and Interviewing Techniques training to help HR with the recruitment process. The training did cover important aspects of job interviews, such as open and closed questions, how to observe the legislation when asking interview questions and most importantly, how to extract the information needed from the candidate. The candidate\u0026rsquo;s skills, abilities and suitability for the job are like a big covered picture and each question aims to reveal a little bit of the picture so that at the end of the interview, the interviewer can make a decision on the suitability of the candidate for the role.\nFrom my experience being on both sides of the table, I believe that filling a job is like buying a house, such that it’s not just the house that matters, but also the neighbourhood and the surrounding area. Whoever is buying a house will double check the area and the neighbourhood quite well because what makes a home is not just the house, but where it exists, the same applies to jobs. What makes a job is not just the job description or responsibilities, but also the interactions and the working environment surrounding the job.\nFrom the candidate\u0026rsquo;s point of view, no matter how attractive the job description is, they don’t know if the work environment, interactions, culture and atmosphere is enjoyable or not until they actually start the job. For all they know the workplace could be suffering from things like lack of professionalism, blame culture, internal politics or bad working practices. From the employer\u0026rsquo;s point of view, no matter how good the candidate is, the employer doesn\u0026rsquo;t know if the candidate is going to be a good fit for their culture and work environment or not until they actually start. The candidate could turn out to be not a good team player or unable to work under pressure to specific deadlines. To mitigate this, the contract usually contains a probation period that has a shorter notice period on both sides.\nUsing my \u0026ldquo;buying a house\u0026rdquo; analogy, this is like buying a house without checking out the surrounding area and then living in it for a couple of months to see what the area is like! and if it turns out not right at the end, you sell and look for another house. Well, people don\u0026rsquo;t usually do that because buying a house is not an easy process, the least to say, it is a very stressful and costly process. Jobs are similar, the candidate has gone through the hassle of updating their CV, getting phone calls whilst at work, having to book time off to go for interviews and trying as much as possible to keep their career move a secret from their current employer. The employer on the other hand, has gone through the hassle of writing a job description, freeing up the resources to conduct the interviews and going through the pain of the recruitment process.\nIn my opinion, it should be a core interview question to ask the candidate to describe their ideal job by elaborating on the following two areas:\nDuties and responsibilities (likes and dislikes). Expected interactions, work environment and culture (likes and dislikes). I believe this missing question will partially address any mismatch of expectations regarding the job interactions, work environment and culture at the interview stage. But, such question is only going to work if the employer is honest throughout the interview. For example it\u0026rsquo;s not good claiming that the company follows an Agile process, when it is actually following a mishmash of processes!, the candidate will find out when they start and will be disappointed. In such a case it\u0026rsquo;s better to say something along the lines of \u0026ldquo;We are trying to implement Agile and are looking for a fresh pair of eyes like yourself to help us improve\u0026rdquo;, but no, during the interview many interviewers will make claims about their working practices which are not entirely true.\nTo make room for such an elaborative question I believe many other less useful questions can be removed from the interview process. Here are examples of such questions:\nWhy are you leaving your current job?, the candidate might genuinely be looking for a career change or a job closer to home, but the reason they are leaving might be related to something negative about their current employer. As candidates are told not to be negative of their current employer, it\u0026rsquo;s guaranteed that most candidates are unlikely to tell the truth when answering this question. Where do you see yourself in X years?, if the purpose of this question is to gauge the suitability of the candidate in the long term or their career aspiration, it\u0026rsquo;s likely it won\u0026rsquo;t serve that purpose most of the times. In X years, if the candidate wants to start their own business, be the CEO of the company or carry on doing the same thing day in and day out, they are not going to say that. I would love to hear your thoughts regarding this, do you believe that questions related to the interactions, working environment and culture will improve the expectations of both the employer and the candidate? are there any other alternatives? I\u0026rsquo;ve come across some initiatives where as part of the interview process the candidate is invited to spend part of a day with the team to get a feel of the work environment, do you think such initiatives can help as well?\n[Thanks to Eliot Kerslake for capturing the LEGO interview with his camera]\n","date":"20 October 2014","externalUrl":null,"permalink":"/2014/10/20/the-missing-interview-question/","section":"Blog","summary":"When it comes to job interviews I’ve had my share throughout the years being on both sides of the table. I’ve conducted my first mock interview years back when my employer at the time sent me on a City \u0026 Guilds…","title":"The Missing Interview Question","type":"posts"},{"content":" \u0026ldquo;Just throw more VMs/resources at it\u0026rdquo;, This is what you will hear time and time again from cloud computing developers, but just how accurate is this statement?\nIn this post we put this statement to the test by running a number of experiments on the Google Cloud Platform. The task is straightforward, use Compute Engine VMs to covert a large volume of data from RDF to CSV format so it can be imported into Bigquery. The dataset is a structured extract from Wikipedia in 64 files totalling 50GB, the files were split between a number of n1-standard-2 instances by running 10 experiments, gradually ramping up VMs from 1 to 10.\nNow, before I go into the results of the experiments let me introduce a few concepts. Usually in computing to speed up the processing of tasks these tasks get distributed between a number of computers so they can be processed in parallel. The hardness of this activity depends on the algorithm being used to process these tasks. Some algorithms are serial in nature and can\u0026rsquo;t be executed in parallel, on the other hand, other algorithms are embarrassingly (entirely) parallel and can be executed independently.\nIn parallel computing it\u0026rsquo;s possible to measure how fast a parallel algorithm runs compared to the correspondent serial algorithm, this fastness is called Speedup. A speedup of a parallel algorithms \\(s_n\\) executed on \\(n\\) VMs is calculated by dividing the runtime \\(t_1\\) when using one VM by the runtime \\(t_n\\) when using \\(n\\) VMs as follow:\n\\[ s_n = \\frac{t_1}{t_n} \\] The ideal or linear speedup is achieved when the number of VMs is proportional with the speed of the algorithm, this happens when the speedup is equal to the number of VMs i.e. \\(s_n = n\\). Another performance metric is Efficiency, which is a value between zero and one to indicate how well the VMs are utilised in executing the parallel algorithm, for example an efficiency of 0.5 (50%) or below isn\u0026rsquo;t good. Efficiency can be calculated as follows:\n\\[ e_n = \\frac{s_n}{n} = \\frac{t_1}{t_n \\times n} \\] Now armed with these concepts, let\u0026rsquo;s see how well our parallel algorithm performed on the large Wikipedia dataset. The chart below shows the linear speedup in blue and the actual speedup in red, as seen from the chart linear speedup can only be achieved when using 2 VMs. Using more VMs results in a gradual decrease in the actual speedup compared to the linear speedup. When the number of VMs reaches 8 the speedup doesn\u0026rsquo;t increase any further, so adding more VMs beyond 8 is simply incurring additional cost, but is not getting the task done any quicker!.\nThe chart below shows the correspondent runtime for both the ideal and actual speedup. For the ideal or linear speedup the runtime continue to reduce when more VMs are added, however the actual runtime hardly decreases beyond 6 VMs.\nTo summarise, from these experiments it\u0026rsquo;s evident that throwing more VMs at tasks doesn\u0026rsquo;t mean they will always get done quicker. This is mainly caused by the complexity and the nature of the tasks, so a better approach than ramping up VMs is to benchmark and optimise the algorithm so it performs better with more VMs.\n","date":"4 June 2014","externalUrl":null,"permalink":"/2014/06/04/throwing-more-vms-at-big-tasks-does-it-work/","section":"Blog","summary":"“Just throw more VMs/resources at it”, This is what you will hear time and time again from cloud computing developers, but just how accurate is this statement?","title":"Throwing more VMs at big tasks, does it work?","type":"posts"},{"content":"","date":"23 March 2014","externalUrl":null,"permalink":"/categories/single-page-apps/","section":"Categories","summary":"","title":"Single Page Apps","type":"categories"},{"content":" What are Single Page Applications (SPAs)? # Single page application is a style of Web applications where the user interface fits in one page, with all the interactions handled by JavaScript on the client side. This approach results in a fluid and dynamic user interface, mainly because the user doesn\u0026rsquo;t have to wait for parts of the UI to load from the server. All the interactions between the UI and the server are handled through AJAX rather than full page navigation. The UI might still load resources from the server as and when they are needed such as HTML fragments, images, fonts, CSS files, JavaScript files, etc\u0026hellip;, but the browser doesn\u0026rsquo;t navigate from the originally loaded page. A good example of an SPA is the Gmail web interface.\nAre SPAs a bleeding edge new concept or a revived old one? # I was recently going over some of my old blog posts and came across this post (Web User Interface from Prototype to Implementation), which made me wonder about many of the fancy technology terms used today and if they are completely new or simply a revival of an old concept, Single Page Apps or SPAs is one of those terms. SPAs are simply Web applications that run on the browser on a single page, where all the user interactions are managed client side and data sent to the server using AJAX. Now how new is this concept?, well it\u0026rsquo;s not new at all, it\u0026rsquo;s probably a recycle and repackage of other old terms, one of which is Rich Internet Applications or RIAs, different names, but the concepts are the same!\nHaving covered most of these concepts during my MSc dissertation research when Web 2.0 was at the peak of its hype, I can just show you how old RIAs or SPAs are. I\u0026rsquo;m going to take you back in history to the year 2007, back then, Dojo, Scriptaculous and Prototype roamed the Web world, JQuery was just about born and AngularJS didn’t even exist. The concepts much hyped about today, such as Client side MVC, MVVM and SPAs weren’t new back then, Dojo ahead of its time offered all of those!, to be honest Dojo had it all:\nSingle Page App pattern Desktop pattern Sophisticated client side MVC, MVVM, etc\u0026hellip; Ultra rich widget library The Dojo 0.9 Demo Form\nAnd to be honest the only reason I think many developers struggled to adopt Dojo at the time was the fact it had a plethora of poor documentations and that many of the concepts it used were ahead of their time.\nThe Concepts back then # Three concepts at the time were key to the development of terms like RIAs and SPAs:\nDuality of the Web - treating the Web as user interface vs. the Web as hypertext system. This is was first published by Jesse Garrett (the guy who came up with the term AJAX) in his book \u0026lsquo;Elements of Reusable User Experience\u0026rsquo;. AJAX - Crane et. al, in his book \u0026lsquo;AJAX in Action\u0026rsquo; site the following as being the key concepts behind using AJAX: The browser hosts an application, not content. The server delivers data, not content. User interaction with the application can be fluid and continuous. This is real coding and requires discipline. (i.e. no cowboy coding ) Web 2.0 - The Web as a Platform, O\u0026rsquo;Reilly\u0026rsquo;s much hyped concept that thought to bring all of the above under one banner, but it was surrounded by hype and overused by businesses at the time to market their products. Rich Internet Application or applications falling under the Web 2.0 category were divided by Forrester Research into three types:\nBrowser based - Applications that use AJAX and run on the browser. Player based - such as applications that run in Flash, Flex or Silverlight. Client based - such Adobe AIR and JavaFX apps. And it all boiled down to having applications that are as rich and interactive as desktop applications.\nAnalogy between RIA and thick clients\nNothing is really new # After reading about all the concepts back then you are probably wondering if anything at all is new in what we have today, well, nothing is really new:\nComplex UI concepts, such as: publish/subscribe notification styles, rich reusable widgets, class definitions, MVVM, etc… were all available at the time. OO JavaScript, JSON and AJAX were there. UI development workflows were available, although based on ANT tools rather than own dedicated tools like Grunt, Bower, etc… And just to show you that nothing is new, here is my diagram of the architecture of a RIA I was designing back in 2007. Most of the concepts of the diagram stood the test of time, JSON, AJAX, client side MVC, etc\u0026hellip;:\nSo why weren’t SPAs adopted back then? # If you remember Google WAVE, the real time collaboration environment from Google that was eventually put to rest triggering many of the technology analysts to simply say it was ahead of its time. Now this is the case with RIAs and SPAs, their concepts were ahead of their time. Many other reasons in my option have meant that adoption of these technologies will take time:\nAmongst all the hype that surrounded Web 2.0 there were simply too many new ground breaking concepts for front end developers to digest as they were still busy trying to learn and use AJAX techniques. The low Internet bandwidth meant rich front end contents were too heavy to load. Browsers weren’t keeping up!. There was no Chrome with V8 supercharge JS engine those days. Because browsers didn’t keep up, developers were busy with Flash, Adobe Air \u0026amp; MS Silverlight. Users weren’t mature enough, still in the ‘Web as Hypertext’ mode. It\u0026rsquo;s just interesting to see how the much hyped Web 2.0 done through the years since 2006, looking at the Gartner Hype Cycle for 2006 and 2009:\nTechnology Hype Cycle 2006 (Source http://www.gartner.com/)\nNow jump to 2009 and see where Web 2.0 is in the hype cycle (it basically means people were finding out that it wasn\u0026rsquo;t as businesses used to pitch it):\nTechnology Hype Cycle 2009 (Source http://www.gartner.com/)\nNow the time is right # Now the time is just about right for RIAs and SPAs for these reasons:\nSuper fast Internet access - rich contents can load much quicker. Super fast Browsers - rich contents render in no time and no longer sluggish during user interactions. Mature end users with high expectations - users nowadays expect a slick and highly responsive UI. Mature developers - developers are now more mature and aware of the concepts mentioned above. Advances in Web technologies such as HTML5 and CSS3 helped to achieve more with less code. Recommendations # Having been involved with Web as a user interface concept at theoretical and practical levels since 2007, and having designed and implemented a few RIAs/SPAs, I can make these recommendations for anyone who wishes to adopt these concepts for mainstream application development:\nFor a production-line of applications, using a UI Widget library (ExtJS, Bootstrap, Polymer, or an in-house library, etc\u0026hellip;) is the best cost effective approach. Separation of Frontend and Backend code. UI end to end testing and backend unit \u0026amp; integrations tests are the way to go. The more complex the UI gets the more starting by just writing code isn’t going to work. Complex UI needs design first, otherwise it’s going to be messy. ","date":"23 March 2014","externalUrl":null,"permalink":"/2014/03/23/single-page-apps-a-bleeding-edge-new-concept-or-a-revived-old-one/","section":"Blog","summary":"Single page application is a style of Web applications where the user interface fits in one page, with all the interactions handled by JavaScript on the client side. This approach results in a fluid and dynamic user…","title":"Single Page Apps - A bleeding edge new concept or a revived old one?","type":"posts"},{"content":"","date":"23 March 2014","externalUrl":null,"permalink":"/categories/web-2.0/","section":"Categories","summary":"","title":"Web 2.0","type":"categories"},{"content":"","date":"23 March 2014","externalUrl":null,"permalink":"/categories/web-and-mobile-technologies/","section":"Categories","summary":"","title":"Web and Mobile Technologies","type":"categories"},{"content":"","date":"3 November 2013","externalUrl":null,"permalink":"/tags/apache-http-components/","section":"Tags","summary":"","title":"Apache Http Components","type":"tags"},{"content":"","date":"3 November 2013","externalUrl":null,"permalink":"/tags/dot/","section":"Tags","summary":"","title":"DOT","type":"tags"},{"content":"","date":"3 November 2013","externalUrl":null,"permalink":"/tags/google-compute-engine/","section":"Tags","summary":"","title":"Google Compute Engine","type":"tags"},{"content":"","date":"3 November 2013","externalUrl":null,"permalink":"/tags/graphvis/","section":"Tags","summary":"","title":"Graphvis","type":"tags"},{"content":"I have recently been interested in Graphviz, an open source graph visualisation software. The graphs are specified in DOT language, which is a plain text graph description language. Just to give you an idea about DOT syntax here is a simple dot script and the equivalent generated graph. Try it Online here:\ndigraph G { a -\u0026gt; b -\u0026gt; c; a-\u0026gt; c; b -\u0026gt; d; c-\u0026gt; d; } Compared to other graph rendering tools, Graphviz is great because it takes care of the graph layout as well as rendering, but if you develop in Java there isn\u0026rsquo;t at the moment a native Java implementation. All the easy to use tools out there require the dot binaries to be present. Now if you develop Java Web apps then generating DOT syntax isn\u0026rsquo;t really a big challenge, rendering it, however is a challenge because you need to invoke the dot command line tool. Wouldn\u0026rsquo;t it be nice to have an Online Graphviz server that you can call with your dot script and it returns back a rendered graph, be it a PNG, PDF or SVG. I mentioned SVG because you can use tools like JQuery Graphviz plugin to render an interactive SVG of your graph where you can easily rearrange, edit or delete the nodes.\nIf you are working with DOT graphs all the time then it makes sense to have the layout and rendering of graphs done by a shared online server that you can simply call from your various applications. Cost is another factor, ideally this should be as cheap as possible to host such a server, this is why I\u0026rsquo;ve thought to try Google Compute Engine specially the f1-micro instance which is very cheap. Now using an f1-micro means I probably want something that is lightweight and runs without the need of a Web container, not even Jetty or Tomcat. So using the HttpCore library from the Apache HttpComponents makes sense for a lightweight elemental HTTP server that invokes the dot command line tool. Here is a simple sequence diagram showing the basic interactions of the server:\nGraphViz Server # Wanting to write as few lines of code as possible and not to re-invent any wheels I\u0026rsquo;ve reused the class from GraphViz Java API and the Elemental Http Server example from the Apache Http Components library to come up with a simple Java based Graphviz Http Server than can bundled as an executable jar and run from the command line, simple!\nDownload the source and a bundled up jar with dependencies from GitHub (https://github.com/omerio/graphviz-server), the port on which the server listens on can be configured as a command line parameter to the jar. To change the default port (8080) edit the DotGraphics.sh in the dist directory:\n#!/bin/sh java -jar DotGraphics.jar 8080 \u0026gt; /dev/null 2\u0026gt;\u0026amp;1 \u0026amp; exit 0 To use the Graphviz server simply submit a HTTP POST with the dot graph script set as the request body. Optionally an output type can be specified on the URL for example:\nPost to http://localhost:8080/svg to render the graph as SVG Post to http://localhost:8080/pdf to render the graph as PDF Post to http://localhost:8080 to render the graph as PNG (default) To test the server use Google Chrome and download the Chrome Poster extension, as you can see the server successfully sends back an SVG:\nSetting up Google Compute Engine # Setting up a GCE instance is very easy and straightforward, go to the Google Compute Engine page and click the try it now link, create a new cloud project, once the project is created click on the Compute Engine link. This requires billing to be enabled so you need a credit card, once done you are ready to create a new instance:\nEnter all the required details and click the \u0026lsquo;Create\u0026rsquo; button.\nModifying the Firewall settings # Once the instance is created you need to navigate to the Networks tab and restrict SSH access to your IP address if you usually connect from a static IP and open up port 8080 for HTTP traffic:\nEnabling SSH # To connect to your instance using SSH simply follow the steps outlined here to install gcutil, then connect to your instance:\n$ gcutil --project=my-project ssh myinst Installing Graphviz binaries # Once you have connected using gcutil it will create a key-pair and set it in the Metadata of your instance. To use normal SSH (Mac) simply just reuse the key generated by gcutil:\n$ ssh -i .ssh/google_compute_engine username@myinst-ipaddress Once you login to the instance you need to install the graphviz binary distribution, then verify dot is installed successfully:\n$ sudo apt-get install graphviz $ dot -v dot - graphviz version 2.26.3 (20100126.1600) Activated plugin library: libgvplugin_pango.so.6 Using textlayout: textlayout:cairo Activated plugin library: libgvplugin_dot_layout.so.6 ... Installing the OpenJDK Java Runtime # Install the OpenJDK headless JRE, once installed verify the installed version of Java:\n$ sudo apt-get install openjdk-7-jre-headless $ java -version java version \u0026#34;1.7.0_25\u0026#34; OpenJDK Runtime Environment (IcedTea 2.3.10) (7u25-2.3.10-1~deb7u1) OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode) SFTP the Graphviz server jar # SFTP the Graphviz server jar to the compute engine instance:\n$ sftp -i ~/.ssh/google_compute_engine username@myinst-ipaddress Connected to ..... sftp\u0026gt; !ls DotGraphics.jar DotGraphics.sh sftp\u0026gt; mput * Now SSH to the instance and run the Graphviz server, verify that it\u0026rsquo;s running by checking the DotGraphics.log\nNow using the Chrome Poster extension verify that you can connect to the server running on the Google Compute Engine instance and can successfully render graphs.\nGoogle App Engine Demo # Here is an AppEngine demo that demonstrates the Graphviz server running on Compute Engine. The source code for the demo is available on Github (https://github.com/omerio/graphviz-appengine). The demo uses the URL Fetch service to post the dot graph supplied by the user to the compute engine instance. The demo uses the JQuery Graphviz plugin to add support for rearranging the nodes.\nhttp://dot-graphics1.appspot.com\n13/10/2015 - Updated link to Graphviz Java API github project. Removed f1-micro pricing as this is likely to change with time\n","date":"3 November 2013","externalUrl":null,"permalink":"/2013/11/03/running-a-graphviz-server-on-google-compute-engine/","section":"Blog","summary":"I have recently been interested in Graphviz, an open source graph visualisation software. The graphs are specified in DOT language, which is a plain text graph description language. Just to give you an idea about DOT…","title":"Running a Java Graphviz server on Google Compute Engine","type":"posts"},{"content":"I was heavily involved in Webapps pen testing \u0026amp; hardening in the past so thought to share some practical tips which Web developers may find useful specially when developing public facing Webapps.\nBelieve it or not looking at the OSI model (Physical, Data link, Network, Transport, Session, Presentation \u0026amp; Application layers) a whopping 80% of the current Internet vulnerabilities/exploits are targeting the Application layer this covers Webapps and others.\nSo how can you secure Webapps? Well, most organisations are using the OWASP top 10 (Open Web Application Security Project) as a guideline of which vulnerabilities to address and penetration test against. The OWASP top 10 highlights the top 10 exploited Webapps vulnerabilities (get the PDF here), how they are exploited and how to reduce the risk associated with them. So the first steps towards securing Webapps is to write secure code. OWASP provide tips related to Java and a secure coding cheat sheet , a must read for Web developers.\nSo is there any other tools/techniques that can be used to secure Webapps, I hear you say? Yes, there are WAS/WAF. That is Web Application Scanners \u0026amp; Web Application Firewalls.\nWeb Application Scanner (WAS) # These are applications that scan your Webapp when it’s offline or online and using a database of all known vulnerabilities hammer your application with potential hacks and provide a nice report on any vulnerabilities found. These tools are quite pricey, but they do a good job. They are like anti-virus software because their exploit signatures DB get updated regularly. You can get a trial version which does a limited number of checks, usually XSS and other vulnerabilities. Here are a couple of the commonly used ones:\nAcunetix Web Security Scanner (http://www.acunetix.com/) NStalker scanner (http://www.nstalker.com/) Web Application Firewall (WAF) # WAF provides active defence for live Webapps, the most widely used WAF with the Apache server is the mod_security module (http://www.modsecurity.org/). It provides first class defence for Webapp by examining request/response data against a set of negative signatures.\nAlso one of the widely used Java security frameworks which is worth having a look at is Spring Security, it’s so comprehensive that it provides extensive security even at Java method invocation level!\nFinally remember, there is no system that is 100% secure, and what is secure today could be insecure tomorrow so security is not just a one off task, it\u0026rsquo;s ongoing. You just do what you can to ensure the risk of exploits is minimized.\n","date":"2 November 2012","externalUrl":null,"permalink":"/2012/11/02/securing-web-applications/","section":"Blog","summary":"I was heavily involved in Webapps pen testing \u0026 hardening in the past so thought to share some practical tips which Web developers may find useful specially when developing public facing Webapps.","title":"Securing Web Applications","type":"posts"},{"content":"","date":"2 November 2012","externalUrl":null,"permalink":"/categories/security/","section":"Categories","summary":"","title":"Security","type":"categories"},{"content":"Apache JMeter is an Open source Java application designed to load test web applications and measure performance. A slight drawback is the limited number of results/graphs bundled with JMeter, but this is easily addressed by installing the JMeter Plugins which adds a wealth of graphs, for example Response times percentiles, Response times vs Threads, etc\u0026hellip;,… and functionality for example Stepping Thread Group to provide an incremental load profile.\nThe Performance Test # Now to the real business, how did I use it?, I have a Web application that runs in Tomcat 6 and connects to MySQL database which I needed to performance test. I utilized the following tools:\nVisualVM Apache JMeter JMeter Plugins Tomcat 6 running locally with APR MySQL 5 running locally The test machine specs are: Intel Core 2 Duo T9300 @ 2.5GHz with 4.00GB RAM.\nThe Results # Initial tests using 50 JMeter users have highlighted a code problem that was blocking a number of Threads. I took a thread dumb using VisualVM during the peak load. The thread dump indicated an inefficient implementation of XML de-serialization which I then revisited and fixed, below is a comparison of response times vs threads before and after the fix:\nBefore\nAfter\nAs seen from both graphs there are clearly some improvements to the response times.\nAnother graph I investigated was the ‘Statistical Aggregate Report’. This graph also shows improvements in response times and throughput after the fix:\nBefore\nAfter\nHowever, the report clearly shows that the response times increase with load whilst the throughput decreases with load, this indicates that the system does not scale very well as it\u0026rsquo;’s. So next step was to try load balancing.\nPerformance Testing with Apache and Tomcat Load Balancing # Ok, next step was to try to load-balance two Tomcat instances using Apache’s mod_proxy, this was tried on a Pentium 4 2.4GHz with 1.5GB RAM running Fedora Core Linux. And the results were promising as the Throughput is now stable, however response times were still increasing with the load profile:\nAnother thread Dumb at peak load have shown that the high response times were due to the blocking caused by the limited number of database connections in the database connections pool. With 50 Active users the database connections pool was set to the maximum of only 10 database connections so most requests reading data from the database had to wait for a free connection.\nNext step is to increase the database connections pool size and to retest.\nTo be continued….\nLast updated: 22/05/2015 - fixed the Jmeter-plugin links\n","date":"22 May 2011","externalUrl":null,"permalink":"/2011/05/22/effective-and-affordable-load-testing-using-jmeter/","section":"Blog","summary":"Apache JMeter is an Open source Java application designed to load test web applications and measure performance. A slight drawback is the limited number of results/graphs bundled with JMeter, but this is easily…","title":"Effective and Affordable Load Testing Using JMeter","type":"posts"},{"content":"","date":"22 May 2011","externalUrl":null,"permalink":"/categories/jmeter/","section":"Categories","summary":"","title":"JMeter","type":"categories"},{"content":"","date":"22 May 2011","externalUrl":null,"permalink":"/categories/performance-testing/","section":"Categories","summary":"","title":"Performance Testing","type":"categories"},{"content":"","date":"22 May 2011","externalUrl":null,"permalink":"/categories/testing/","section":"Categories","summary":"","title":"Testing","type":"categories"},{"content":"","date":"20 February 2010","externalUrl":null,"permalink":"/categories/javascript/","section":"Categories","summary":"","title":"JavaScript","type":"categories"},{"content":" Introduction # How do we implement the User Interface to edit a list of complex objects, some of these objects might have a list of other objects as well?, there are a number of techniques to implement this, I present three of them below. You need to assess the usability and ease of implementation for each one before choosing a technique for your UI.\nThe easiest single view technique # I believe this is the easier and the technique that provide the least rich user experience.\nHave a drop down list with an identifier for all the object with a new button next to it. When the user selects an item from the list an area is displayed with the fields populated with the object properties including a Save and Delete buttons. Clicking the Add New button results in the area displayed with blank fields\nExample 1: # A drop down list with button to create a new record\nClicking the \u0026lsquo;New\u0026rsquo; button displays empty fields\nSelecting an existing record from the drop down list shows the fields prepopulated to provide the ability to edit the record\nExample 2: # This example shows a more complex UI using the Dojo Toolkit to edit a collection of objects with each object having another collection of objects:\nSelecting a row from the multiple select shows the second level of editable objects\nThe rich multi-view technique # This technique is highly effective in a MDI (Multiple Document Interface) view. But can also be used in a classical Web view where each view is displayed on a separate Web page. This technique is highly effective when editing complex object graphs specially when a new view is launched to edit or add a new object as shown in example 2 below.\nDisplay all the details in a grid with new, save and delete buttons. The Grid is readonly. Selecting a record populates a form on the side or below the grid or pops a window with the details, after editing the user can click save. Clicking add new can either launch a window with a blank form or show a blank form to be edited\nExample 1: # The example below shows a form binding to a grid, when the user makes a selection from the grid the form is pre-populated with the selected grid record. The form could also be used to add new records to the grid\nExample 2: # The example below shows a grid with a number of buttons to edit/delete and create a new record\nClicking the edit button launches a new window to edit the record details\nClicking the new button launches a new window to add a new record\nThe rich single view technique # Have a fully editable grid with save, new and delete buttons. All operations are performed on the Grid. I believe this technique provides a very rich user experience if it utilises AJAX to update the data once the user has finished updating the records. The view can also fit within a single Web page view. The technique may only useful when editing objects that have simple properties, but not objects that have a collection of other objects that also need to be editable.\nExample 1: # Example 2: # ","date":"20 February 2010","externalUrl":null,"permalink":"/2010/02/20/three-techniques-to-implement-a-web-ui-for-editing-a-list-of-objects/","section":"Blog","summary":"How do we implement the User Interface to edit a list of complex objects, some of these objects might have a list of other objects as well?, there are a number of techniques to implement this, I present three of them…","title":"Three techniques to implement a Web UI for editing a list of objects","type":"posts"},{"content":"","date":"24 June 2008","externalUrl":null,"permalink":"/tags/database-theory/","section":"Tags","summary":"","title":"Database Theory","type":"tags"},{"content":"","date":"24 June 2008","externalUrl":null,"permalink":"/categories/er-model/","section":"Categories","summary":"","title":"ER Model","type":"categories"},{"content":"","date":"24 June 2008","externalUrl":null,"permalink":"/tags/er-model/","section":"Tags","summary":"","title":"ER Model","type":"tags"},{"content":"","date":"24 June 2008","externalUrl":null,"permalink":"/categories/jpa/","section":"Categories","summary":"","title":"JPA","type":"categories"},{"content":"","date":"24 June 2008","externalUrl":null,"permalink":"/tags/jpa/","section":"Tags","summary":"","title":"JPA","type":"tags"},{"content":"Java Persistence API (JPA) coined with Java 5 Annotations made Object to Relational Mapping (ORM) a bliss. The problem is most of the detailed documentations available usually consider JPA from the POJOs perspective, but what if we are creating a JPA application that should persist to an existing database. An Entity Relationship diagram can easily be constructed for existing databases using a tool such as MySQL Workbench. From the ER Model we can work backwards to create the JPA annotations required to persist to the existing database. This article provides a mapping that can be used to create the JPA annotations for an existing database\u0026rsquo;s ER Model using Hibernate\u0026rsquo;s implementation of JPA.\nI prefer to use the crow’s feet notation in the ER model to represent the cordiality of the relations and the participation conditions as explained in Table 1 below. An example of an ER model that uses this notation is shown in Figure 1 below.\nTable 1 – Crow’s feet notation used in the ER- modelling\nSymbol Meaning Open blob - Zero or one participation Closed blob - Exactly one participation Open blob and crow’s foot - Zero or more participation Closed blob and crow’s foot - One or more participation Figure 1 - Sample ER Model using crow\u0026rsquo;s feet annotation\nThe approach that can be used to map relations form the E-R model into JPA annotations is summarised in Table 2 below. This approach can be followed to map the entity classes into the existing data schema using JPA annotations. Good database design should resolve all the Many to Many relationships during the logical database design stage by introducing a third dependent relation, and hence there was no need to use @ManyToMany JPA mapping.\nI have also discovered that the Cascade annotation for foreign key updates on the parent table should not be included in the child entity as this will result in the deletion of the parent row whenever a child row is deleted. It was clear that this constraint should be implemented in the database schema and not included in the JPA annotation as it does not have the desired effect.\nTable 2 – ER-Model and Relational Mode to JPA and Hibernate mappings\nER-Model Relational Mode JPA Annotation Entity Relation @Entity Identifier Primary key @Id Alternate Key @UniqueConstraint(columnNames = {\u0026ldquo;staff_id\u0026rdquo;, \u0026ldquo;holiday_year\u0026rdquo;}) Relationships: One to One Primary key + Foreign key mechanism – plus declaring the Foreign key as alternate key. Mandatory participation condition: Left side – can be achieved using a constraint. Right side – not allowing null for Foreign key. @OneToOne Declaring alternate key: @UniqueConstraint(columnNames = {\u0026ldquo;staff_id\u0026rdquo;, \u0026ldquo;holiday_year\u0026rdquo;}) Declaring a check constraint: @org.hibernate.annotations.Check( constraints = \u0026ldquo;(mileage is not null) or (amount is not null)\u0026rdquo; ) Declaring not null: @Column(nullable = false) One to Many Primary key + Foreign key mechanism. Mandatory Participation condition: Left side – can be achieved using a constraint. Right side – not allowing null for Foreign key. @OneToMany annotation created in the owning class Declaring a check constraint: @org.hibernate.annotations.Check( constraints = \u0026ldquo;(mileage is not null) or (amount is not null)\u0026rdquo; ) Declaring not null: @Column(nullable = false) Cascading: Cascading works well when annotating the owning relationship @OneToMany(cascade = {CascadeType.ALL}) Many to One Primary key + Foreign key mechanism. Mandatory Participation condition: Left side – not allowing null for Foreign key. Right side – can be achieved using a constraint. @ManyToOne annotation created in the owned class Declaring a check constraint: @org.hibernate.annotations.Check( constraints = \u0026ldquo;(mileage is not null) or (amount is not null)\u0026rdquo; ) Declaring not null: @Column(nullable = false) Cascading: Cascading annotation should not be used in the child class, as will result in the primary key table being updated. Should be included in the database schema Many to Many Should be resolved at the Logical database design, by resolving the M-N relationships into 3 relations Mandatory Participation condition: Implemented as above for One to Many relationships on the One side of the two owning relations @ManyToMany The expected name for the intermediate table is table1_table2 unless specified otherwise in the annotation. ","date":"24 June 2008","externalUrl":null,"permalink":"/2008/06/24/mapping-er-model-into-jpa-annotations/","section":"Blog","summary":"Java Persistence API (JPA) coined with Java 5 Annotations made Object to Relational Mapping (ORM) a bliss. The problem is most of the detailed documentations available usually consider JPA from the POJOs perspective…","title":"Mapping ER-Model into JPA Annotations","type":"posts"},{"content":" I\u0026rsquo;m Omer Dawelbeit. My passion for computers and programming started as a hobby almost 3 decades ago when I started learning GW-BASIC, FORTRAN, Borland Pascal, Delphi and C++. This passion has evolved over the years to cover Full-stack software development, Web and Mobile technologies, Cloud Computing, Big Data, IoT and Machine Learning.\nIn addition to technology I\u0026rsquo;m also passionate about people development, community building, mentoring and building effective high performance teams. My approach to leadership, building things and solving problems is highly logical and grounded in both knowledge and experience. My quest for knowledge is the reason I\u0026rsquo;ve pursed academic qualifications (e.g. Masters and PhD) and professional certifications.\nCurrently I work as a Principal Solutions Architect with Amazon Web Services. I work with public sector customers on their strategic initiatives, helping them with cloud adoption and architecting their solutions on AWS. Prior to joining AWS, I worked for McKinsey and Company, where I led large digital transformations in government and banking in the UK, GCC and South East Asia. I\u0026rsquo;ve 20+ years of experience leading technology on large digital transformations for customers of all sizes across multiple verticals and geographies. I hold a PhD in Computer Science from the University of Reading in the UK.\nAcademic Publications # Efficient Dictionary Compression for Processing RDF Big Data Using Google BigQuery, Omer Dawelbeit and Rachel McCrindle. 2016 IEEE Global Communications Conference (GLOBECOM), Washington, USA CloudEx, A Novel Cloud-based Task Execution Framework, Omer Dawelbeit and Rachel McCrindle, 2016 IEEE Global Communications Conference: Workshops: Fifth International Workshop on Cloud Computing Systems, Networks, and Applications (GC16 Workshops CCSNA), Washington, USA A Novel Cloud Based Elastic Framework for Big Data Preprocessing, Omer Dawelbeit and Rachel McCrindle, Proceeding of the Sixth Computer Science and Electronic Engineering Conference 2014 (CEEC'14), IEEE Open source projects: # I was at some point the contributor and maintainer of the following open source projects:\nECARF, Elastic cloud based reasoning framework for the Semantic Web, http://ecarf.io CloudEx, A generic elastic cloud based framework for the execution of embarrassingly parallel tasks. http://cloudex.io. Appstart, Graphviz-Server, etc\u0026hellip; more projects on GitHub Developer community (2015 - 2021): # I was a Google Developer Expert for Cloud and was also the co-organizer for the Reading and Thames Valley Google Developers Group. I have grown the group through technical community engagement and collaboration. I enjoy mentoring \u0026amp; inspiring others on digital technologies, sharing knowledge through open source code samples, presentations and blog posts. Below is a brief summary of my past community engagements.\nCommunity Groups\n2017 - 2021 - Co-organiser of the Google Developer Group Cloud Emirates 2015 - 2016 - Co-organiser of the Reading \u0026amp; Thames Valley Google Developer Group Events Timeline\n2017 - 2019 - Various Google Cloud talks and roadshows in Saudi, UAE, Kuwait and Algeria. Various visits to Google Cloud Next Conferences and GDE Summits 23rd Sep 2017 - Talk \u0026amp; Codelab - Introduction to Google Cloud Platform - Google Developer Roadshow Cairo. Smart Village Cairo, Egypt 6th Sep 2017 - Training - Attending the Sprint Master Academy, Krakow Poland. 5th Sep 2017 - Community - Attending Google Developer Days Europe in Krakow, Poland. Helping with Cloud Platform office hours. 4th Sep 2017 - GDE Community - Attending the Google Developers Expert Summit in Krakow, Poland. 22nd Aug 2017 - Talk - Introduction to Google Cloud - GDG Cloud Emirates. Dubai Internet City. 17th Jul 2017 - Community - Co-organiser of GDG Cloud Emirates. UAE 22nd Apr 2017 - Talk - Introduction to Google Cloud - GDG Jeddah in Umm Al Qura University in Mekkah, Saudi Arabia. 8th Dec 2016 - Conference - Presenting a paper titled \u0026lsquo;Efficient Dictionary Compression for Processing RDF Big Data Using Google BigQuery\u0026rsquo; at the IEEE Globecom 2016 conference in Washington DC. 29th Jul 2016 - Talk - Introduction to Google App Engine - GDG North East (Newcastle). 23/24 Mar 2016 - Live correspondence from Google Cloud Platform Next 2016 in San Francisco with both GDG Reading \u0026amp; Thames Valley \u0026amp; GDG North East 4th Feb 2016 - Talk - Google Container Engine and Kubernetes - Oxford GDG Meetup 3rd Mar 2016 - Talk - Introduction to Google App Engine - Oxford GDG Meetup 30th Jan 2016 - Mentor - Mentoring participants @ Hack Cambridge hackathon as part of the Google team 27th Jan 2016 - Talk - Cloud And Social Are All You Need To Go Big - Digital Henley Meetup 13th Jan 2016 - Talk - Introduction to Google App Engine - Reading \u0026amp; Thames Valley GDG Meetup Dec / Jan 2016 - My blog posts on Kubernetes, Cloud Shell, Pi Zero with App Engine and Google Container Engine were featured in many newsletters, here are a few: Pi Zero post Featured on Adafruit blog and on the Google Cloud Platform Flipboard piweekly newsletter issue 137 Google Cloud Platform News Roundup for December 2015 LinuxBSDos.com - A 10 minutes guide to Kubernetes Cloud Development Weekly issue 34 Web Operations Weekly issue 47 DevOps Newsletter issue 253 Devops Weekly #260 Dockone.io translated to Chinese Kubernetes Daily 5th Aug 2015 - Talk - Google Bigquery, processing big data without the infrastructure - Reading \u0026amp; Thames Valley GDG developer gathering. 21st Oct 2015 - Conference presentation - A Novel Cloud Based Elastic Framework for Big Data Preprocessing - 7th Computer Science \u0026amp; Electronic Engineering Conference Links: # Twitter: https://twitter.com/omerio GitHub: https://github.com/omerio LinkedIn: https://www.linkedin.com/in/omerdawelbeit/ StackOverflow: http://stackoverflow.com/users/1958841/omerio ","date":"24 June 2008","externalUrl":null,"permalink":"/omer-dawelbeit/","section":"Technology with Passion","summary":"I’m Omer Dawelbeit. My passion for computers and programming started as a hobby almost 3 decades ago when I started learning GW-BASIC, FORTRAN, Borland Pascal, Delphi and C++. This passion has evolved over the years to…","title":"About Me","type":"page"},{"content":" Introduction # One approach to user interface design is the use of prototypes. Prototypes are incomplete versions of the user interface for the software being developed. Using prototypes at the early stages of requirement gathering and analysis helps in clarifying the requirements with the client and forms the starting point for the user interface design and implementation. In this article we look at the prototyping approach to user interface development for Rich Internet Applications. User interface prototypes are usually developed to look informal, but yet functional. The problem with developing a complete GUI for prototype is the fact that it gives the client the impression that large parts of the development is already done and leads them later to question why the development is taking such a long time when almost all of the application was done during the requirement gathering. A good article on this is the “Don\u0026rsquo;’t make the Demo look Done”, from Kathy Sierra’s blog. Below is an example of two user interfaces, one that looks almost done and the other one look extremely informal by using the Napkin look and feel for Java applications.\nFigure 1. Prototype that looks almost done. Figure 1. Prototype that looks almost done.\nFigure 2. Prototype that looks informal Figure 2. Prototype that looks informal\nAn established approach to software design is the use of the USDP (Unified Software Development Process). The main focus of the USDP is on modelling the business requirements using standards such as UML. This approach reinforces the focus on business first then technology, enabling the business model to exist independent of any platform or technology. The prototype is usually developed during the preliminary system design stage, which aims at providing an analysis model.\nRequirement Capture and Modelling # Typical steps followed during the requirement capture and modelling stage are summarised below:\nTechniques # Requirement elicitation using methods such as: Background reading Fact finding interviews with the client Observation Document sampling Questionnaires. Use case modelling to be carried out and documented as follows: Use case diagram. Use case description One or more flow of events (i.e. Normal or baseline flow, alternative flows and exception flows) Activity diagram to aid the understanding of the use case. Activity diagrams are less technical than sequence diagrams so their use at such an early stage can aid the understanding of the business stakeholders. Initial system architecture can be developed to help guide subsequent steps during the development process and can be refined and adjusted as the development process progresses. The initial system architecture is usually the package structure of the system. Prototypes of some key user interfaces are to be produced to aid the requirement understanding and gathering. Key Deliverables # Use case model Requirements list Initial architecture Prototypes Prototyping for the web # When developing prototypes for the Web a distinction have to be made be made between web sites and web applications. Web sites are mainly concerned about information; on the other hand web applications are concerned with tasks such as the steps involved in a process. This basic duality of the web was discussed by Jesse James in his book “The Elements of User Experience”. The web can basically be treated as a software interface or as a hypertext system and software developed for the web can categorised as either an application or a hypertext information space or a hybrid of the two. In this article I would use the term websites to refer to applications that utilize the web as a hypertext system and the term web applications to refer to applications that utilize the web as a software interface or a hybrid of both.\nBased on new concepts such as Web 2.0 I will go a step further and divide web applications into two categories:\nTraditional web applications based on web pages where the progress in the application is largely based on form submission or Hyperlink navigation to post or retrieve data/contents from the server. Rich Internet Applications based on using AJAX or some other technology to submit or retrieve data from the server without having to use forms or submit a whole page. These applications resemble desktop thick clients in their interactivity, responsiveness, flexibility and rich user interface. RIA can be divided into two types: In-Browser or browser based – These utilize the browser capabilities such as HTML, JavaScript, CSS and XMLHTTPRequest. Out-of-browser or plug-in based – These utilize the capabilities of external plugins such as Adobe AIR, Microsoft’s Silverlight and JavaFX from Sun. Prototyping for traditional web applications: # For traditional web applications prototyping can be done using paper or computer based wire-frames. Microsoft Power Point, Visio or Open Office Impress can be used to construct these wire-frames; in fact any image authoring tool can be used to draw up a prototype. But such prototypes are simply static and lack the interactivity. DENIM is a tool that can be used to create prototypes for websites and web applications. Such prototypes do look informal and can be designed to have some elements of interactivity.\nFigure 3. DENIM prototype for web application\nPrototyping for RIA # As Rich Internet Applications closely resembles traditional desktop applications there is nothing that stops us from using the same prototyping methods used for the desktop clients to design prototypes for RIA. The user interface for RIA runs entirely on the client side, but the UI is first transferred from the server to the client upon first request. For subsequent requests the client only exchanges data with the server. RIAs now benefit from a large number of rich widgets that sometimes look even better than widgets available for traditional thick clients; this analogy between these two types of applications is shown below. But before starting to prototype for RIAs we probably need to have a catalogue of currently used visual widgets such as controls, commands, pointers and windows. A good starting place is the Visual Index on the Microsoft Window Desktop Apps design guide. We can then short-list the widgets that can be included in the prototype and the completed user interface later on.\nFigure 4. Analogy between RIA and thick clients\nMany tools can be used to prototype for RIAs, I personally prefer using the NetBeans Swing GUI builder but you can also use other Visual IDEs such as Visual Basic IDE. NetBeans GUI builder is based on Java Swing so the Napkin look and feel can easily be integrated to create informal prototypes. The builder has a good size palette of widgets such as tab containers, grids, panels, buttons ,etc\u0026hellip;\nFigure 5. Swing UI builder\nPrototype Example # Below is a prototype developed for a window that deals with the user’s tasks. Each user has a number of tasks that they can view; the user can refresh all the tasks, delete a task or set a task as completed. The prototype was first developed in NetBeans builder then exported to Paint to customize the look and feel of the action buttons.\nFigure 6. Modified Swing GUI for tasks window\nBelow is the use case diagram for the view tasks use case.\nFigure 7. Use case diagram for tasks requirements\nChoosing the user interface development toolkit # You could write HTML, JavaScript and CSS to create a RIA, but this will be time consuming and why invent the wheel when there are many free toolkits and frameworks out there with many features to offer. Below is a summary of the widely used toolkits/frameworks to develop the user interface for Rich Internet Applications\nToolkit Vendor Type Technologies Adobe AIR Adobe Plugin JavaScript, AJAX, Flex or Flash Silverlight Microsoft Plugin JavaScript, C#, VB.NET, AJAX, IronPython or IronRuby JavaFX Sun Plugin JavaFX Dojo Toolkit Dojo Foundation In browser DHTML, JavaScript and CSS Yahoo! UI Library Yahoo In browser DHTML, JavaScript and CSS. Google Web Toolkit Google In browser Java DWR GetaHead In browser Java, etc\u0026hellip; And many more\u0026hellip; User Interface Design using MVC # Below I will explain the implementation of the prototype in Figure 6 above into a fully fledged user interface using the Dojo Toolkit and the MVC pattern. The dojo toolkit streamlines JavaScript development by providing custom rich widgets, many utility functions and an Ajax library. JavaScript can be used as an object oriented language based on prototyping, the dojo toolkit provide a Java like style to defining objects and inheritance in JavaScript.\nFigure 8. Client side MVC GUI\nWidget objects defined in Dojo can also have a HTML template and linked to DOM nodes. The widget object then manipulates the HTML nodes to display information or responds to user actions such as button clicks, etc…\u0026hellip;\nA new approach to web user interface development # The approach I have followed here to develop the rich HTML user interface is not widely used for web applications, although the process used is deeply rooted in the USDP and was used for a long time to model user interface for desktop applications developed in languages such as Java and Visual Basic. Saying this, the fact that JavaScript supports object orientation, there nothing that prevents us from using the same methodology to design the user interface for rich internet applications. These steps are based on the methodology outlined by Bennett et al in their book “Object Oriented Systems Analysis and Design Using UML”:\nPrototyping the user interface. Designing the classes. Modelling the interaction involved in the interface using interaction or communication diagrams. Modelling the control of the interface using state machines for complex UI components. View – Presentation (HTML, CSS and DOM objects)… # The view on the client side user interface is created either as HTML or JavaScript Widgets. HTML view is created when widgets are not required to provide functionality depending on user input or when it’s owned by a JavaScript Widget. On the other hand JavaScript widgets are used for complex user interface components such as widgets that control other widgets to perform a specific operation as shown above for the “My Tasks” widget. Each JavaScript widget consists of the following:\nJavaScript object, I use the term View Support Object (VSO) to describe these JavaScript Objects as they control the DOM nodes attached to them (Figure 8). DOM nodes to represent the HTML template for the widget. Can also include other child widgets as widgets can be nested. For each view component the view is implemented using this widget approach as follow:\nDefine the widget template which consists of HTML, CSS and other widgets, this is defined in a HTML file e.g. manageTasks.html. Define the VSO for the widget in a JavaScript file, e.g. manageTasks.js using a JavaScript class definition from the Dojo toolkit as follows: To define a widget class (supports Mixins, multiple inheritance): dojo.widget.defineWidget(\u0026#34;ClassName\u0026#34;, [SuperClass1, SuperClass2, …], { property1: “”, property2: “”, method1: function() { // method code here } } To define a non-widget class (supports Mixins, multiple inheritance): dojo.declare(\u0026#34;ClassName\u0026#34;,[SuperClass1, SuperClass2, …], { property1: “”, property2: “”, // acts as a Java constructor initializer : function(urls) { this.urls = urls; this.user = \u0026#34;\u0026#34;; this.ajaxTimeOut = 600; // 600s, 10mins to wait for ajax // calls }, method1: function() { // method code here } } Listing 1. Dojo classes declaration\nImplement the methods required for the VSO to control the DOM nodes. These methods follow from the user interface models such as boundary classes and interaction diagrams as shown in the diagrams below. For the ManageTasks widget a boundary class diagram and an interaction diagram was developed to detail the functionality of the ManageTasks JavaScript widget. If I treat JavaScript as Object oriented language then there is nothing that stops me Modelling Java alongside JavaScript Using a UML CASE tool as shown in the sequence diagram below showing client and server side objects. JavaScript built-in types can also be modelled using a js.lang namespace or package, the same applies to the classes provided by the toolkit being used. Figure 9. Modelling JavaScript using UML\nFigure 10. Boundary Class Diagram for ManageTasks Widget\nThe sequence diagram below demonstrates the modelling of server and client components together. The blue object denotes client side JavaScript object whilst the green one symbolise the server side classes. As shown the call queryAll is an asynchronous (AJAX) call as symbolised by the open arrow message and the queryAllTasksLoad call the call back method provided for AJAX response.\nFigure 11. Sequence diagram for View Task use case on the ManageTasks widget\nModel – Data # Using JSON the server side domain model can be serialized and transferred to the client in JSON format, it will then be build into client side JavaScript objects using the eval method. This way the model on the client side will reflect the objects on the server side and there is no need to create class definition for the model on the client side.\nAn example is the sequence diagram shown above, the server side object TaskWrapper is serialized into JSON format and delivered to the client side, so the same object graph will also exist on the client side. The main advantage of this approach is the fact that there is no need to maintain two sets of class definitions for TaskWrapper on the client and server side, as the server side definition can be used in both layers.\nController # The controller orchestrates the view, the model and facilitates communications with the server to retrieve/save data using Ajax. The controller does the following:\nHolds reference to and controls all the JavaScript widgets. Queries data from the server using Ajax and reconstructs the response into JavaScript model objects. Saves data to the server from the updated model. Enforces security and authorisation on the client side. The same enforcements are done on the server side to ensure no JavaScript tampering was done. The controller can be programmed to use the JavaScript prototype or the dojo.declare to inherit from a generic controller class, which defines common functionality such as dirty flags, etc…. The controller can have methods such as: Show, Hide, Delete, Create, Save, Retrieve, etc\u0026hellip;\nThe Final User Interface # The final user interface for the task management window is shown below:\nFigure 12. The data is loading using AJAX\nFigure 13. The data is loaded and displayed\nFigure 14. The user make a selection.\nReferences\nArrington, C.T. and Rayhan, S.H. (2003). Enterprise Java with UML. 2nd ed. Wiley Bennett, S. McRobb, S. and Farmer, R. (2006). Object-oriented systems analysis and design using UML. 3rd ed. London: McGraw-Hill Companies. Bennett, S. Skelton, J. Lunn, K. (2004). Schaum’s Outline UML. 2nd ed. McGraw-Hill. Dojo Ajax Toolkit. URL: http://dojotoolkit.org/ Dojo Toolkit Object oriented concepts and inheritance. URL: http://dojotoolkit.org/book/dojo-book-0-4/part-3-dojo-programming-model/object-oriented-concepts-and-inheritance Gadge, V. Technology options for Rich Internet Applications. URL: http://www-128.ibm.com/developerworks/library/wa-richiapp/ [16 September 2007] Garrett, J. J. (2002). The Elements of User Experience: User-Centered Design for the Web. New Riders Press. JavaScript Object Notation. URL: http://www.json.org/ O’Reilly, T. What Is Web 2.0, Design Patterns and Business Models for the Next Generation of Software. URL: http://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.html?page=1[16 September 2007] ","date":"9 March 2008","externalUrl":null,"permalink":"/2008/03/09/web-user-interface-from-prototype-to-implementation/","section":"Blog","summary":"One approach to user interface design is the use of prototypes. Prototypes are incomplete versions of the user interface for the software being developed. Using prototypes at the early stages of requirement gathering…","title":"Web User Interface from Prototype to Implementation","type":"posts"},{"content":"","externalUrl":null,"permalink":"/authors/","section":"Authors","summary":"","title":"Authors","type":"authors"},{"content":"","externalUrl":null,"permalink":"/series/","section":"Series","summary":"","title":"Series","type":"series"}]